2024-09-21T14:11:12.110 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T14:11:12.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T14:11:12.222 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909929' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd cache max dirty: 0 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4623 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 80/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATemBgD1awZxYY4oEDXshWP0benAYdv4h2PCgE5fnHD1BUfB7qTIspOXfLh5KERlUcigsuKutrVDnyBMJdUK2w= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHhDk/uCjD4mo2q5T0Px8nYodV4mxydcWs3fA43NZMkR51jG1eGqi17tpDxEkqrrHAAhDJ+PgaeALI9RU703yM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_22:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T14:11:12.222 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa; will attempt to use it 2024-09-21T14:11:12.223 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa/tasks 2024-09-21T14:11:12.223 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T14:11:12.224 INFO:teuthology.task.internal:Checking packages... 2024-09-21T14:11:12.244 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-21T14:11:12.244 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T14:11:12.244 INFO:teuthology.packaging:ref: None 2024-09-21T14:11:12.244 INFO:teuthology.packaging:tag: None 2024-09-21T14:11:12.244 INFO:teuthology.packaging:branch: reef 2024-09-21T14:11:12.244 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:11:12.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-21T14:11:12.485 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-871.g6a1bdf85 2024-09-21T14:11:12.487 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T14:11:12.494 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T14:11:12.495 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T14:11:12.542 INFO:teuthology.task.internal:Saving configuration 2024-09-21T14:11:12.555 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T14:11:12.575 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T14:11:12.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 14:06:06.476355', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATemBgD1awZxYY4oEDXshWP0benAYdv4h2PCgE5fnHD1BUfB7qTIspOXfLh5KERlUcigsuKutrVDnyBMJdUK2w='} 2024-09-21T14:11:12.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 14:06:06.477680', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHhDk/uCjD4mo2q5T0Px8nYodV4mxydcWs3fA43NZMkR51jG1eGqi17tpDxEkqrrHAAhDJ+PgaeALI9RU703yM='} 2024-09-21T14:11:12.623 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T14:11:12.634 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-21T14:11:12.634 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-21T14:11:12.634 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T14:11:12.699 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc08bc0fa30>, signals=[15]) 2024-09-21T14:11:12.699 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T14:11:12.709 INFO:teuthology.task.internal:Opening connections... 2024-09-21T14:11:12.710 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:11:12.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:11:12.784 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:11:12.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:11:12.857 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T14:11:12.869 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-09-21T14:11:12.888 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-09-21T14:11:12.888 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-09-21T14:11:12.945 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T14:11:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T14:11:12.947 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T14:11:12.947 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-09-21T14:11:12.974 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-09-21T14:11:12.993 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-09-21T14:11:12.993 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-09-21T14:11:13.050 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T14:11:13.051 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T14:11:13.052 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T14:11:13.052 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-09-21T14:11:13.074 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T14:11:13.088 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T14:11:13.101 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T14:11:13.102 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-09-21T14:11:13.104 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-09-21T14:11:13.121 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T14:11:13.131 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T14:11:13.132 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-09-21T14:11:13.161 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-09-21T14:11:13.232 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T14:11:13.300 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T14:11:13.319 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T14:11:13.319 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T14:11:13.320 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T14:11:13.320 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T14:11:13.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:11:13.337 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-09-21T14:11:13.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:11:13.338 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-21T14:11:13.394 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-503.el9.x86_64 2024-09-21T14:11:13.394 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.14.0-503.el9.x86_64 2024-09-21T14:11:13.394 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2024-09-21T14:11:13.395 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-503.el9.x86_64 2024-09-21T14:11:13.396 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.14.0-503.el9.x86_64 2024-09-21T14:11:13.396 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum install -y kernel 2024-09-21T14:11:16.347 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.3 MB 00:01 2024-09-21T14:11:16.710 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.3 MB 00:01 2024-09-21T14:11:18.064 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-21T14:11:18.503 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T14:11:22.962 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-21T14:11:23.382 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-21T14:11:24.455 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-21T14:11:24.849 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-21T14:11:25.700 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T14:11:25.745 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T14:11:31.106 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-21T14:11:31.184 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-21T14:11:33.004 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T14:11:33.004 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T14:11:33.065 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T14:11:33.066 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T14:11:33.108 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:11:33.117 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:11:33.118 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:11:33.119 INFO:teuthology.orchestra.run.smithi073.stdout:Install 4 Packages 2024-09-21T14:11:33.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:11:33.119 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 79 M 2024-09-21T14:11:33.119 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 126 M 2024-09-21T14:11:33.119 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:11:33.159 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:11:33.169 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:11:33.169 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-09-21T14:11:33.169 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:11:33.169 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:11:33.169 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout:Install 4 Packages 2024-09-21T14:11:33.170 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:11:33.171 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 79 M 2024-09-21T14:11:33.171 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 126 M 2024-09-21T14:11:33.171 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:11:33.402 INFO:teuthology.orchestra.run.smithi161.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 353 kB/s | 36 kB 00:00 2024-09-21T14:11:33.657 INFO:teuthology.orchestra.run.smithi073.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 124 kB/s | 36 kB 00:00 2024-09-21T14:11:33.786 INFO:teuthology.orchestra.run.smithi161.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-21T14:11:34.237 INFO:teuthology.orchestra.run.smithi161.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-21T14:11:34.503 INFO:teuthology.orchestra.run.smithi161.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-21T14:11:34.503 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:11:34.504 INFO:teuthology.orchestra.run.smithi161.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-21T14:11:34.957 INFO:teuthology.orchestra.run.smithi073.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 9.8 MB/s | 16 MB 00:01 2024-09-21T14:11:35.183 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:11:35.531 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:11:35.531 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:11:35.891 INFO:teuthology.orchestra.run.smithi073.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-21T14:11:36.492 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:11:36.492 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:11:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:11:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T14:11:39.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:11:40.866 INFO:teuthology.orchestra.run.smithi073.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 4.0 MB/s | 29 MB 00:07 2024-09-21T14:11:40.866 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:11:40.866 INFO:teuthology.orchestra.run.smithi073.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-21T14:11:41.548 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:11:41.886 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:11:41.887 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:11:42.492 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:11:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:11:42.855 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:11:42.858 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:11:45.318 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:11:46.059 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T14:11:46.141 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:11:47.437 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:11:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:11:48.874 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:11:49.001 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:11:52.392 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:11:53.850 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:11:53.942 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:11:58.807 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:30.819 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:30.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T14:12:30.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:12:30.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:12:31.657 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:12:31.999 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T14:12:32.132 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:32.132 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T14:12:32.132 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T14:12:32.132 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T14:12:32.132 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T14:12:32.132 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T14:12:32.132 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T14:12:32.133 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T14:12:32.191 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:32.191 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T14:12:32.192 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2024-09-21T14:12:32.223 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2024-09-21T14:12:32.223 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2024-09-21T14:12:32.223 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.06 2024-09-21T14:12:32.223 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 82.el9 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Group : Unspecified 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 8274670 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : builder@centos.org 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T14:12:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:12:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T14:12:32.226 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T14:12:32.226 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 5.14.0-508.el9.x86_64 2024-09-21T14:12:32.226 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T14:12:32.920 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-09-21T14:12:34.186 INFO:teuthology.orchestra.run.smithi161.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T14:12:34.203 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-09-21T14:12:34.205 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-09-21T14:12:34.220 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.F3l2aXQJJB 2024-09-21T14:12:34.221 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.F3l2aXQJJB 2024-09-21T14:12:34.287 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.F3l2aXQJJB 2024-09-21T14:12:34.429 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.F3l2aXQJJB is 6KB 2024-09-21T14:12:34.517 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.F3l2aXQJJB 2024-09-21T14:12:34.532 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T14:12:34.594 INFO:teuthology.orchestra.run.smithi161.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T14:12:34.595 INFO:teuthology.orchestra.run.smithi161.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T14:12:34.595 INFO:teuthology.orchestra.run.smithi161.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T14:12:34.595 INFO:teuthology.orchestra.run.smithi161.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T14:12:34.596 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T14:12:34.755 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-09-21T14:12:37.042 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:37.081 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T14:12:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T14:12:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T14:12:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T14:12:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:12:38.303 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:12:38.663 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T14:12:38.739 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.740 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T14:12:38.740 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T14:12:38.740 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T14:12:38.740 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T14:12:38.740 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T14:12:38.740 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T14:12:38.740 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T14:12:38.799 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T14:12:38.799 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T14:12:38.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -qi grub2-tools 2024-09-21T14:12:38.831 INFO:teuthology.orchestra.run.smithi073.stdout:Name : grub2-tools 2024-09-21T14:12:38.831 INFO:teuthology.orchestra.run.smithi073.stdout:Epoch : 1 2024-09-21T14:12:38.831 INFO:teuthology.orchestra.run.smithi073.stdout:Version : 2.06 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Release : 82.el9 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: x86_64 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Group : Unspecified 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Size : 8274670 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:License : GPLv3+ 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Packager : builder@centos.org 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:Vendor : CentOS 2024-09-21T14:12:38.832 INFO:teuthology.orchestra.run.smithi073.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:Summary : Support tools for GRUB. 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:Description : 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:hardware devices. 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:12:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T14:12:38.834 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T14:12:38.834 INFO:teuthology.task.kernel:Updating grub on smithi073 to boot 5.14.0-508.el9.x86_64 2024-09-21T14:12:38.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T14:12:39.584 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-09-21T14:12:40.976 INFO:teuthology.orchestra.run.smithi073.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T14:12:41.007 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-09-21T14:12:41.010 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-21T14:12:41.025 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.XREdXYyJQ1 2024-09-21T14:12:41.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XREdXYyJQ1 2024-09-21T14:12:41.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.XREdXYyJQ1 2024-09-21T14:12:42.543 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.XREdXYyJQ1 is 6KB 2024-09-21T14:12:42.567 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.XREdXYyJQ1 2024-09-21T14:12:42.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T14:12:42.653 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T14:12:42.653 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T14:12:42.653 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T14:12:42.653 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T14:12:42.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T14:12:42.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-09-21T14:13:04.766 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T14:13:04.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:13:04.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:13:12.824 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T14:13:12.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-21T14:13:12.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:13:39.439 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-09-21T14:13:48.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:13:48.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:14:06.835 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-09-21T14:14:12.830 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T14:14:18.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:14:18.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:14:19.359 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-09-21T14:14:19.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:14:19.792 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T14:14:19.792 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-09-21T14:14:19.806 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-508.el9.x86_64 2024-09-21T14:14:19.806 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T14:14:19.806 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T14:14:19.806 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T14:14:20.807 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T14:14:20.807 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T14:14:20.875 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-09-21T14:14:20.900 DEBUG:teuthology.parallel:result is None 2024-09-21T14:14:21.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-21T14:14:21.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:14:22.266 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-21T14:14:22.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-21T14:14:22.693 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T14:14:22.693 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-21T14:14:22.708 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-508.el9.x86_64 2024-09-21T14:14:22.708 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T14:14:22.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T14:14:22.708 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T14:14:23.709 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T14:14:23.709 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T14:14:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-09-21T14:14:23.826 DEBUG:teuthology.parallel:result is None 2024-09-21T14:14:23.826 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T14:14:23.834 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T14:14:23.834 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T14:14:23.836 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T14:14:23.852 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T14:14:23.874 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T14:14:23.907 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T14:14:23.907 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T14:14:23.909 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T14:14:23.936 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T14:14:23.942 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T14:14:23.942 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T14:14:23.974 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T14:14:24.001 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:14:24.005 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:14:24.012 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:14:24.015 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:14:24.017 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T14:14:24.024 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T14:14:24.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T14:14:24.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T14:14:24.082 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T14:14:24.090 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T14:14:24.090 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T14:14:24.123 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T14:14:24.137 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:14:24.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:14:24.262 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:14:24.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:14:24.385 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:14:24.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T14:14:24.450 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:14:24.494 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:14:24.539 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:14:24.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:14:24.660 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:14:24.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T14:14:24.724 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-09-21T14:14:24.726 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-09-21T14:14:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T14:14:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T14:14:25.096 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T14:14:25.104 INFO:teuthology.task.internal:Starting timer... 2024-09-21T14:14:25.105 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T14:14:25.115 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T14:14:25.122 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-21T14:14:25.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-09-21T14:14:25.160 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-09-21T14:14:25.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-09-21T14:14:25.287 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-09-21T14:14:25.291 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T14:14:25.291 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-09-21T14:14:25.314 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-09-21T14:14:25.314 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-09-21T14:14:25.351 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-09-21T14:14:25.352 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-09-21T14:14:25.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T14:14:25.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:14:25.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T14:14:25.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:14:25.440 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T14:14:25.440 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-09-21T14:14:25.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-09-21T14:14:25.722 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T14:14:25.731 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T14:14:25.734 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T14:14:25.838 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T14:14:25.847 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T14:14:25.848 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T14:18:07.032 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-09-21T14:18:07.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-21T14:18:07.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:18:07.108 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-21T14:18:07.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-21T14:18:07.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:18:07.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:18:07.263 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-09-21T14:18:07.341 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T14:18:07.341 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T14:18:07.358 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T14:18:07.358 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T14:18:07.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:18:07.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T14:18:07.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:18:07.388 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T14:18:07.402 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T14:18:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T14:18:07.423 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T14:18:07.427 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-09-21T14:18:07.438 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-09-21T14:18:07.447 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-09-21T14:18:07.452 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T14:18:07.458 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-09-21T14:18:07.467 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T14:18:07.472 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T14:18:07.486 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T14:18:07.529 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.532 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.547 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.552 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:18:07.552 INFO:teuthology.run_tasks:Running task install... 2024-09-21T14:18:07.559 DEBUG:teuthology.task.install:project ceph 2024-09-21T14:18:07.560 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-21T14:18:07.560 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-21T14:18:07.560 INFO:teuthology.task.install:Using flavor: default 2024-09-21T14:18:07.565 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T14:18:07.565 INFO:teuthology.task.install:extra packages: [] 2024-09-21T14:18:07.565 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-21T14:18:07.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:18:07.567 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-21T14:18:07.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:18:07.703 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-21T14:18:07.703 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-21T14:18:07.712 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-21T14:18:07.712 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-21T14:18:07.822 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T14:18:07.822 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:18:07.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T14:18:07.824 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T14:18:07.824 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:18:07.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T14:18:07.851 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T14:18:07.851 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T14:18:07.851 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T14:18:07.852 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T14:18:07.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T14:18:07.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T14:18:07.991 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T14:18:07.992 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T14:18:08.055 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-09-21T14:18:08.057 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-09-21T14:18:08.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-09-21T14:18:08.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-09-21T14:18:08.284 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-09-21T14:18:08.288 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-09-21T14:18:08.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw 2024-09-21T14:18:08.316 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw 2024-09-21T14:18:08.857 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 294 kB/s | 77 kB 00:00 2024-09-21T14:18:08.876 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 281 kB/s | 77 kB 00:00 2024-09-21T14:18:09.124 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 51 kB/s | 11 kB 00:00 2024-09-21T14:18:09.160 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-21T14:18:09.383 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-21T14:18:09.427 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-21T14:18:10.194 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T14:18:11.192 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T14:18:15.695 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-21T14:18:16.834 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-21T14:18:20.262 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.5 MB 00:00 2024-09-21T14:18:21.722 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-21T14:18:22.435 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T14:18:22.501 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T14:18:23.928 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-21T14:18:24.740 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T14:18:28.107 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-21T14:18:30.037 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T14:18:30.358 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:18:30.374 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:18:30.374 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:30.374 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T14:18:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T14:18:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:30.377 INFO:teuthology.orchestra.run.smithi161.stdout:Install 28 Packages 2024-09-21T14:18:30.378 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 2 Packages 2024-09-21T14:18:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:30.378 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 53 M 2024-09-21T14:18:30.378 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:18:31.158 INFO:teuthology.orchestra.run.smithi161.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-21T14:18:31.217 INFO:teuthology.orchestra.run.smithi161.stdout:(2/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 432 kB/s | 25 kB 00:00 2024-09-21T14:18:31.309 INFO:teuthology.orchestra.run.smithi161.stdout:(3/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 7.6 MB/s | 712 kB 00:00 2024-09-21T14:18:31.434 INFO:teuthology.orchestra.run.smithi161.stdout:(4/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-21T14:18:31.459 INFO:teuthology.orchestra.run.smithi161.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 3.1 MB/s | 477 kB 00:00 2024-09-21T14:18:31.509 INFO:teuthology.orchestra.run.smithi161.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 899 kB/s | 45 kB 00:00 2024-09-21T14:18:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:(7/30): python3-ceph-common-18.2.4-855.g19e2a2b 1.7 MB/s | 129 kB 00:00 2024-09-21T14:18:31.652 INFO:teuthology.orchestra.run.smithi161.stdout:(8/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 21 MB/s | 4.5 MB 00:00 2024-09-21T14:18:31.677 INFO:teuthology.orchestra.run.smithi161.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 1.7 MB/s | 162 kB 00:00 2024-09-21T14:18:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 6.2 MB/s | 321 kB 00:00 2024-09-21T14:18:31.727 INFO:teuthology.orchestra.run.smithi161.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-21T14:18:31.761 INFO:teuthology.orchestra.run.smithi161.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-21T14:18:31.928 INFO:teuthology.orchestra.run.smithi161.stdout:(13/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 15 MB/s | 18 MB 00:01 2024-09-21T14:18:32.020 INFO:teuthology.orchestra.run.smithi161.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 495 kB/s | 45 kB 00:00 2024-09-21T14:18:32.045 INFO:teuthology.orchestra.run.smithi161.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 337 kB/s | 107 kB 00:00 2024-09-21T14:18:32.071 INFO:teuthology.orchestra.run.smithi161.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 518 kB/s | 160 kB 00:00 2024-09-21T14:18:32.113 INFO:teuthology.orchestra.run.smithi161.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-21T14:18:32.138 INFO:teuthology.orchestra.run.smithi161.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T14:18:32.163 INFO:teuthology.orchestra.run.smithi161.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T14:18:32.213 INFO:teuthology.orchestra.run.smithi161.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-21T14:18:32.247 INFO:teuthology.orchestra.run.smithi161.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 820 kB/s | 26 kB 00:00 2024-09-21T14:18:32.272 INFO:teuthology.orchestra.run.smithi161.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T14:18:32.304 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:18:32.306 INFO:teuthology.orchestra.run.smithi161.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2024-09-21T14:18:32.321 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:32.321 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:18:32.321 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:32.321 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:18:32.321 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T14:18:32.322 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T14:18:32.323 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T14:18:32.324 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout:Install 28 Packages 2024-09-21T14:18:32.325 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 2 Packages 2024-09-21T14:18:32.326 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:32.326 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 53 M 2024-09-21T14:18:32.326 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:18:32.331 INFO:teuthology.orchestra.run.smithi161.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-21T14:18:32.365 INFO:teuthology.orchestra.run.smithi161.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T14:18:32.398 INFO:teuthology.orchestra.run.smithi161.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-21T14:18:32.457 INFO:teuthology.orchestra.run.smithi161.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T14:18:32.532 INFO:teuthology.orchestra.run.smithi161.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T14:18:32.599 INFO:teuthology.orchestra.run.smithi161.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 17 MB/s | 3.3 MB 00:00 2024-09-21T14:18:32.683 INFO:teuthology.orchestra.run.smithi161.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-21T14:18:32.688 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:18:32.688 INFO:teuthology.orchestra.run.smithi161.stdout:Total 23 MB/s | 53 MB 00:02 2024-09-21T14:18:32.757 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:18:32.793 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:18:32.793 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:18:33.272 INFO:teuthology.orchestra.run.smithi073.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-21T14:18:33.339 INFO:teuthology.orchestra.run.smithi073.stdout:(2/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 378 kB/s | 25 kB 00:00 2024-09-21T14:18:33.430 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:18:33.430 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:18:33.473 INFO:teuthology.orchestra.run.smithi073.stdout:(3/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-21T14:18:33.515 INFO:teuthology.orchestra.run.smithi073.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 4.0 MB/s | 712 kB 00:00 2024-09-21T14:18:33.540 INFO:teuthology.orchestra.run.smithi073.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 7.0 MB/s | 477 kB 00:00 2024-09-21T14:18:33.599 INFO:teuthology.orchestra.run.smithi073.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 770 kB/s | 45 kB 00:00 2024-09-21T14:18:33.691 INFO:teuthology.orchestra.run.smithi073.stdout:(7/30): python3-ceph-common-18.2.4-855.g19e2a2b 1.4 MB/s | 129 kB 00:00 2024-09-21T14:18:33.750 INFO:teuthology.orchestra.run.smithi073.stdout:(8/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 19 MB/s | 4.5 MB 00:00 2024-09-21T14:18:33.933 INFO:teuthology.orchestra.run.smithi073.stdout:(9/30): ceph-common-18.2.4-855.g19e2a2be.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T14:18:33.959 INFO:teuthology.orchestra.run.smithi073.stdout:(10/30): python3-cephfs-18.2.4-855.g19e2a2be.el 606 kB/s | 162 kB 00:00 2024-09-21T14:18:33.984 INFO:teuthology.orchestra.run.smithi073.stdout:(11/30): python3-rados-18.2.4-855.g19e2a2be.el9 1.3 MB/s | 321 kB 00:00 2024-09-21T14:18:34.018 INFO:teuthology.orchestra.run.smithi073.stdout:(12/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 3.5 MB/s | 300 kB 00:00 2024-09-21T14:18:34.043 INFO:teuthology.orchestra.run.smithi073.stdout:(13/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.2 MB/s | 99 kB 00:00 2024-09-21T14:18:34.152 INFO:teuthology.orchestra.run.smithi073.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-21T14:18:34.177 INFO:teuthology.orchestra.run.smithi073.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 555 kB/s | 107 kB 00:00 2024-09-21T14:18:34.202 INFO:teuthology.orchestra.run.smithi073.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-09-21T14:18:34.253 INFO:teuthology.orchestra.run.smithi073.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-21T14:18:34.295 INFO:teuthology.orchestra.run.smithi073.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T14:18:34.328 INFO:teuthology.orchestra.run.smithi073.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 1.4 MB/s | 111 kB 00:00 2024-09-21T14:18:34.362 INFO:teuthology.orchestra.run.smithi073.stdout:(20/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-21T14:18:34.421 INFO:teuthology.orchestra.run.smithi073.stdout:(21/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.4 MB/s | 308 kB 00:00 2024-09-21T14:18:34.446 INFO:teuthology.orchestra.run.smithi073.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-21T14:18:34.471 INFO:teuthology.orchestra.run.smithi073.stdout:(23/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-09-21T14:18:34.496 INFO:teuthology.orchestra.run.smithi073.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T14:18:34.530 INFO:teuthology.orchestra.run.smithi073.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T14:18:34.588 INFO:teuthology.orchestra.run.smithi073.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T14:18:34.647 INFO:teuthology.orchestra.run.smithi073.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T14:18:34.731 INFO:teuthology.orchestra.run.smithi073.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T14:18:34.823 INFO:teuthology.orchestra.run.smithi073.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-21T14:18:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:18:34.881 INFO:teuthology.orchestra.run.smithi073.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-21T14:18:34.886 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:18:34.886 INFO:teuthology.orchestra.run.smithi073.stdout:Total 21 MB/s | 53 MB 00:02 2024-09-21T14:18:34.917 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-21T14:18:34.946 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-21T14:18:34.958 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:18:34.993 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:18:34.993 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:18:34.996 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-21T14:18:35.312 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-21T14:18:35.411 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:35.504 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:35.604 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-21T14:18:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:35.641 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:18:35.641 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:18:35.710 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:35.746 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-21T14:18:35.790 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-21T14:18:35.861 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-21T14:18:35.891 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:35.957 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:36.062 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-21T14:18:36.090 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-21T14:18:36.139 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-21T14:18:36.171 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-21T14:18:36.198 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-21T14:18:36.234 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-21T14:18:36.267 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-21T14:18:36.577 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-21T14:18:36.630 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-21T14:18:36.720 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-21T14:18:36.751 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-21T14:18:36.787 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-21T14:18:37.012 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:18:37.103 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-21T14:18:37.139 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-21T14:18:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-21T14:18:37.196 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-21T14:18:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-21T14:18:37.288 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-21T14:18:37.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-21T14:18:37.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-21T14:18:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-21T14:18:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:37.704 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:37.796 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-21T14:18:37.822 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:37.885 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:37.938 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-21T14:18:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-21T14:18:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-21T14:18:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-21T14:18:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-21T14:18:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-21T14:18:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-21T14:18:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-21T14:18:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-21T14:18:38.468 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-21T14:18:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-21T14:18:38.822 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-21T14:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-21T14:18:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-21T14:18:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-21T14:18:39.394 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-21T14:18:39.425 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-21T14:18:39.489 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-21T14:18:39.530 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-21T14:18:39.565 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-21T14:18:40.096 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-21T14:18:40.240 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-21T14:18:40.240 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T14:18:40.241 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T14:18:40.241 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:40.269 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:41.774 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:42.306 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-21T14:18:42.345 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-21T14:18:42.445 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-21T14:18:42.445 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T14:18:42.446 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T14:18:42.446 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:42.479 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-09-21T14:18:50.196 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:50.916 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:18:50.942 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:50.944 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T14:18:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T14:18:51.019 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-09-21T14:18:52.245 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-09-21T14:18:52.246 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-21T14:18:52.271 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-21T14:18:52.272 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:52.273 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-21T14:18:52.274 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-21T14:18:52.971 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:18:52.996 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:52.998 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T14:18:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T14:18:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:53.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.388 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T14:18:53.389 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:53.390 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:18:53.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-test 2024-09-21T14:18:54.245 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:54.245 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-21T14:18:54.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-21T14:18:54.247 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-21T14:18:54.248 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-21T14:18:54.248 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-21T14:18:54.248 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-21T14:18:54.339 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:26 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:18:54.476 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-21T14:18:54.477 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout:Install 4 Packages 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:18:54.478 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 40 M 2024-09-21T14:18:54.479 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 164 M 2024-09-21T14:18:54.479 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:18:54.751 INFO:teuthology.orchestra.run.smithi161.stdout:(1/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 748 kB/s | 166 kB 00:00 2024-09-21T14:18:55.126 INFO:teuthology.orchestra.run.smithi161.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-21T14:18:55.147 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T14:18:55.147 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:55.147 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-09-21T14:18:55.147 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T14:18:55.148 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.149 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:55.150 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 476 kB/s | 304 kB 00:00 2024-09-21T14:18:55.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-test 2024-09-21T14:18:55.818 INFO:teuthology.orchestra.run.smithi161.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-21T14:18:55.820 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:18:55.820 INFO:teuthology.orchestra.run.smithi161.stdout:Total 30 MB/s | 40 MB 00:01 2024-09-21T14:18:55.848 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:18:55.868 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:18:55.868 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:18:56.110 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:26 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:18:56.201 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:18:56.202 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:18:56.243 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:18:56.243 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:56.243 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-21T14:18:56.244 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout:Install 4 Packages 2024-09-21T14:18:56.245 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:18:56.246 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 40 M 2024-09-21T14:18:56.246 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 164 M 2024-09-21T14:18:56.247 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:18:56.521 INFO:teuthology.orchestra.run.smithi073.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-21T14:18:56.563 INFO:teuthology.orchestra.run.smithi073.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T14:18:56.621 INFO:teuthology.orchestra.run.smithi073.stdout:(3/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 695 kB/s | 166 kB 00:00 2024-09-21T14:18:56.674 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:18:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T14:18:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T14:18:56.844 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-21T14:18:57.663 INFO:teuthology.orchestra.run.smithi073.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-21T14:18:57.665 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:18:57.666 INFO:teuthology.orchestra.run.smithi073.stdout:Total 28 MB/s | 40 MB 00:01 2024-09-21T14:18:57.694 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:18:57.714 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:18:57.715 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:18:58.046 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:18:58.047 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:18:58.531 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:18:58.615 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T14:18:58.663 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T14:18:58.690 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-21T14:19:00.579 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-21T14:19:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-21T14:19:02.354 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-21T14:19:02.430 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-21T14:19:02.619 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-21T14:19:02.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-21T14:19:02.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-21T14:19:02.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:02.906 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:03.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph 2024-09-21T14:19:03.787 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:35 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:03.922 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-21T14:19:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T14:19:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T14:19:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout:Install 33 Packages 2024-09-21T14:19:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:03.928 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 29 M 2024-09-21T14:19:03.928 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 104 M 2024-09-21T14:19:03.928 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:04.551 INFO:teuthology.orchestra.run.smithi161.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 56 kB/s | 6.5 kB 00:00 2024-09-21T14:19:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-21T14:19:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-21T14:19:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-21T14:19:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T14:19:04.727 INFO:teuthology.orchestra.run.smithi161.stdout:(2/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 7.2 MB/s | 2.1 MB 00:00 2024-09-21T14:19:04.769 INFO:teuthology.orchestra.run.smithi161.stdout:(3/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.4 MB/s | 1.5 MB 00:00 2024-09-21T14:19:04.869 INFO:teuthology.orchestra.run.smithi161.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.4 MB/s | 246 kB 00:00 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:04.894 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:04.936 INFO:teuthology.orchestra.run.smithi161.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-21T14:19:05.053 INFO:teuthology.orchestra.run.smithi161.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T14:19:05.095 INFO:teuthology.orchestra.run.smithi161.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T14:19:05.137 INFO:teuthology.orchestra.run.smithi161.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-21T14:19:05.162 INFO:teuthology.orchestra.run.smithi161.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-21T14:19:05.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph 2024-09-21T14:19:05.199 INFO:teuthology.orchestra.run.smithi161.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.0 MB/s | 125 kB 00:00 2024-09-21T14:19:05.224 INFO:teuthology.orchestra.run.smithi161.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.4 MB/s | 215 kB 00:00 2024-09-21T14:19:05.391 INFO:teuthology.orchestra.run.smithi161.stdout:(12/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_ 25 MB/s | 17 MB 00:00 2024-09-21T14:19:05.475 INFO:teuthology.orchestra.run.smithi161.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 139 kB/s | 35 kB 00:00 2024-09-21T14:19:05.534 INFO:teuthology.orchestra.run.smithi161.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 359 kB/s | 51 kB 00:00 2024-09-21T14:19:05.559 INFO:teuthology.orchestra.run.smithi161.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-21T14:19:05.601 INFO:teuthology.orchestra.run.smithi161.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 429 kB/s | 172 kB 00:00 2024-09-21T14:19:05.626 INFO:teuthology.orchestra.run.smithi161.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-21T14:19:05.660 INFO:teuthology.orchestra.run.smithi161.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-21T14:19:05.685 INFO:teuthology.orchestra.run.smithi161.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 206 kB/s | 11 kB 00:00 2024-09-21T14:19:05.710 INFO:teuthology.orchestra.run.smithi161.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 763 kB/s | 18 kB 00:00 2024-09-21T14:19:05.735 INFO:teuthology.orchestra.run.smithi161.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 478 kB/s | 23 kB 00:00 2024-09-21T14:19:05.761 INFO:teuthology.orchestra.run.smithi161.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-21T14:19:05.778 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:35 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:05.786 INFO:teuthology.orchestra.run.smithi161.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-21T14:19:05.811 INFO:teuthology.orchestra.run.smithi161.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-21T14:19:05.845 INFO:teuthology.orchestra.run.smithi161.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-21T14:19:05.870 INFO:teuthology.orchestra.run.smithi161.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-09-21T14:19:05.895 INFO:teuthology.orchestra.run.smithi161.stdout:(27/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-21T14:19:05.919 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:05.921 INFO:teuthology.orchestra.run.smithi161.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-21T14:19:05.921 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-21T14:19:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T14:19:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T14:19:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:05.925 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:05.926 INFO:teuthology.orchestra.run.smithi073.stdout:Install 33 Packages 2024-09-21T14:19:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:05.926 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 29 M 2024-09-21T14:19:05.926 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 104 M 2024-09-21T14:19:05.927 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:05.947 INFO:teuthology.orchestra.run.smithi161.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-21T14:19:05.971 INFO:teuthology.orchestra.run.smithi161.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 472 kB/s | 36 kB 00:00 2024-09-21T14:19:05.997 INFO:teuthology.orchestra.run.smithi161.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-21T14:19:06.022 INFO:teuthology.orchestra.run.smithi161.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T14:19:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-09-21T14:19:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:Total 14 MB/s | 29 MB 00:02 2024-09-21T14:19:06.166 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:06.204 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:06.204 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:06.248 INFO:teuthology.orchestra.run.smithi073.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-21T14:19:06.440 INFO:teuthology.orchestra.run.smithi073.stdout:(2/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-21T14:19:06.499 INFO:teuthology.orchestra.run.smithi073.stdout:(3/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 5.7 MB/s | 2.1 MB 00:00 2024-09-21T14:19:06.582 INFO:teuthology.orchestra.run.smithi073.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.9 MB/s | 246 kB 00:00 2024-09-21T14:19:06.606 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:06.606 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:06.700 INFO:teuthology.orchestra.run.smithi073.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 10 MB/s | 4.7 MB 00:00 2024-09-21T14:19:06.816 INFO:teuthology.orchestra.run.smithi073.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-21T14:19:07.000 INFO:teuthology.orchestra.run.smithi073.stdout:(7/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-21T14:19:07.025 INFO:teuthology.orchestra.run.smithi073.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 510 kB/s | 106 kB 00:00 2024-09-21T14:19:07.076 INFO:teuthology.orchestra.run.smithi073.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-21T14:19:07.126 INFO:teuthology.orchestra.run.smithi073.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-21T14:19:07.151 INFO:teuthology.orchestra.run.smithi073.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-21T14:19:07.176 INFO:teuthology.orchestra.run.smithi073.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 766 kB/s | 135 kB 00:00 2024-09-21T14:19:07.202 INFO:teuthology.orchestra.run.smithi073.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-21T14:19:07.227 INFO:teuthology.orchestra.run.smithi073.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-09-21T14:19:07.253 INFO:teuthology.orchestra.run.smithi073.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 674 kB/s | 51 kB 00:00 2024-09-21T14:19:07.278 INFO:teuthology.orchestra.run.smithi073.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-21T14:19:07.328 INFO:teuthology.orchestra.run.smithi073.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-21T14:19:07.353 INFO:teuthology.orchestra.run.smithi073.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-21T14:19:07.379 INFO:teuthology.orchestra.run.smithi073.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-21T14:19:07.404 INFO:teuthology.orchestra.run.smithi073.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-21T14:19:07.429 INFO:teuthology.orchestra.run.smithi073.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-09-21T14:19:07.450 INFO:teuthology.orchestra.run.smithi073.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 429 kB/s | 19 kB 00:00 2024-09-21T14:19:07.466 INFO:teuthology.orchestra.run.smithi073.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 536 kB/s | 20 kB 00:00 2024-09-21T14:19:07.483 INFO:teuthology.orchestra.run.smithi073.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-21T14:19:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-21T14:19:07.534 INFO:teuthology.orchestra.run.smithi073.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-21T14:19:07.551 INFO:teuthology.orchestra.run.smithi073.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 388 kB/s | 16 kB 00:00 2024-09-21T14:19:07.576 INFO:teuthology.orchestra.run.smithi073.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-21T14:19:07.602 INFO:teuthology.orchestra.run.smithi073.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-09-21T14:19:07.635 INFO:teuthology.orchestra.run.smithi073.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-21T14:19:07.660 INFO:teuthology.orchestra.run.smithi073.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-21T14:19:07.686 INFO:teuthology.orchestra.run.smithi073.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T14:19:07.711 INFO:teuthology.orchestra.run.smithi073.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-21T14:19:07.715 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:07.715 INFO:teuthology.orchestra.run.smithi073.stdout:Total 16 MB/s | 29 MB 00:01 2024-09-21T14:19:07.831 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:07.865 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:07.869 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:07.869 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:07.925 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-21T14:19:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-21T14:19:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-21T14:19:08.107 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-21T14:19:08.177 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-21T14:19:08.218 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-21T14:19:08.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-21T14:19:08.299 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-21T14:19:08.307 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:08.308 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:08.334 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-21T14:19:08.367 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-21T14:19:08.408 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-21T14:19:08.443 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-21T14:19:08.477 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-21T14:19:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-21T14:19:08.589 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-21T14:19:08.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-21T14:19:08.832 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-21T14:19:08.893 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-21T14:19:08.931 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-21T14:19:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-21T14:19:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-21T14:19:09.222 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-21T14:19:09.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-21T14:19:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-21T14:19:09.461 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-21T14:19:09.501 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-21T14:19:09.541 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:09.614 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-21T14:19:09.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-21T14:19:09.646 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-21T14:19:09.704 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-21T14:19:09.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-21T14:19:09.794 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-21T14:19:09.796 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-21T14:19:09.820 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-21T14:19:09.820 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:09.820 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T14:19:09.820 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T14:19:09.820 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T14:19:09.821 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:09.874 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-21T14:19:09.915 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-21T14:19:09.958 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-21T14:19:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-21T14:19:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-21T14:19:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-21T14:19:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-21T14:19:10.132 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-21T14:19:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-21T14:19:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-21T14:19:10.295 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-21T14:19:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-21T14:19:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-21T14:19:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-21T14:19:10.621 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-21T14:19:10.670 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-21T14:19:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-21T14:19:10.913 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-21T14:19:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-21T14:19:11.046 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-21T14:19:11.144 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-21T14:19:11.184 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-21T14:19:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-21T14:19:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-21T14:19:11.290 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:11.290 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T14:19:11.290 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T14:19:11.290 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T14:19:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:11.300 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-21T14:19:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-21T14:19:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-21T14:19:11.511 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-21T14:19:11.511 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:11.511 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T14:19:11.511 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T14:19:11.512 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T14:19:11.512 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:11.764 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-21T14:19:11.790 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-21T14:19:11.790 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:11.790 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T14:19:11.790 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T14:19:11.791 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T14:19:11.791 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:12.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-21T14:19:12.030 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-21T14:19:12.031 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:12.031 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T14:19:12.031 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T14:19:12.031 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T14:19:12.031 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:12.105 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-21T14:19:12.999 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T14:19:13.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:13.474 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T14:19:13.498 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:13.700 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T14:19:13.724 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:13.787 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-21T14:19:13.881 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-21T14:19:13.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-21T14:19:13.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-21T14:19:13.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-21T14:19:13.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-21T14:19:13.883 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-21T14:19:13.884 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-21T14:19:13.884 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-21T14:19:13.884 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-21T14:19:13.884 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-21T14:19:13.885 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-21T14:19:13.885 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-21T14:19:13.885 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-21T14:19:14.908 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-21T14:19:14.908 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T14:19:14.909 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T14:19:14.910 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:14.911 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:15.279 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-base 2024-09-21T14:19:15.525 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-21T14:19:15.526 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-21T14:19:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-21T14:19:15.528 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-21T14:19:15.528 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-21T14:19:15.529 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-21T14:19:15.529 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-21T14:19:15.529 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-21T14:19:15.529 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-21T14:19:15.529 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-21T14:19:15.870 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:47 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:15.958 INFO:teuthology.orchestra.run.smithi161.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:16.000 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:16.001 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:16.001 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:16.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install cephadm 2024-09-21T14:19:16.569 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-21T14:19:16.569 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:16.569 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T14:19:16.570 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T14:19:16.571 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:16.572 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:16.645 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:48 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:16.775 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:16.776 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:16.776 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:16.776 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:16.776 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:16.776 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 226 k 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 217 k 2024-09-21T14:19:16.777 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:16.983 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-base 2024-09-21T14:19:17.006 INFO:teuthology.orchestra.run.smithi161.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 988 kB/s | 226 kB 00:00 2024-09-21T14:19:17.007 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:17.007 INFO:teuthology.orchestra.run.smithi161.stdout:Total 982 kB/s | 226 kB 00:00 2024-09-21T14:19:17.007 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:17.012 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:17.012 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:17.015 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:17.015 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:17.088 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:17.520 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:17.572 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:47 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:17.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:17.661 INFO:teuthology.orchestra.run.smithi073.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:17.702 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:17.703 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:19:17.703 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:17.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install cephadm 2024-09-21T14:19:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:18.225 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:18.351 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:48 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:18.436 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T14:19:18.482 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:18.483 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 226 k 2024-09-21T14:19:18.484 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 217 k 2024-09-21T14:19:18.484 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:18.701 INFO:teuthology.orchestra.run.smithi073.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-21T14:19:18.701 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:18.702 INFO:teuthology.orchestra.run.smithi073.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-21T14:19:18.702 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:18.707 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:18.707 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:18.710 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:18.710 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:18.778 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:19.033 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:51 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:19.173 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:19.174 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 143 k 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 395 k 2024-09-21T14:19:19.175 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:19.230 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:19.329 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:19.387 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 674 kB/s | 143 kB 00:00 2024-09-21T14:19:19.387 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:19.388 INFO:teuthology.orchestra.run.smithi161.stdout:Total 669 kB/s | 143 kB 00:00 2024-09-21T14:19:19.388 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:19.395 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:19.395 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:19.452 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:19.452 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:19.712 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:19.713 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:19.713 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T14:19:19.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:19.925 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:20.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T14:19:20.583 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:20.583 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:20.583 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:20.584 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:20.584 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:20.584 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:20.755 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:50 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:20.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr 2024-09-21T14:19:20.891 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:20.892 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:20.893 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:19:20.893 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:20.893 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 143 k 2024-09-21T14:19:20.893 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 395 k 2024-09-21T14:19:20.893 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:21.091 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 721 kB/s | 143 kB 00:00 2024-09-21T14:19:21.092 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:21.092 INFO:teuthology.orchestra.run.smithi073.stdout:Total 715 kB/s | 143 kB 00:00 2024-09-21T14:19:21.092 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:21.099 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:21.099 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:21.156 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:21.156 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:21.398 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:21.408 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:21.409 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T14:19:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:21.493 INFO:teuthology.orchestra.run.smithi161.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:21.535 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:21.536 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:21.536 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:21.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T14:19:22.181 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:22.315 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:22.315 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:22.315 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:22.315 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:22.315 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout:Install 5 Packages 2024-09-21T14:19:22.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:22.317 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 3.8 M 2024-09-21T14:19:22.317 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 87 M 2024-09-21T14:19:22.317 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:22.348 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:22.507 INFO:teuthology.orchestra.run.smithi161.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 166 kB/s | 24 kB 00:00 2024-09-21T14:19:22.540 INFO:teuthology.orchestra.run.smithi161.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 91 kB/s | 16 kB 00:00 2024-09-21T14:19:22.591 INFO:teuthology.orchestra.run.smithi161.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 367 kB/s | 31 kB 00:00 2024-09-21T14:19:22.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr 2024-09-21T14:19:22.624 INFO:teuthology.orchestra.run.smithi161.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T14:19:22.774 INFO:teuthology.orchestra.run.smithi161.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 8.5 MB/s | 3.5 MB 00:00 2024-09-21T14:19:22.778 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:22.778 INFO:teuthology.orchestra.run.smithi161.stdout:Total 8.2 MB/s | 3.8 MB 00:00 2024-09-21T14:19:22.778 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:22.785 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:22.785 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:22.869 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:22.869 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:23.118 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:23.211 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:23.213 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-21T14:19:23.254 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-21T14:19:23.283 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-21T14:19:23.300 INFO:teuthology.orchestra.run.smithi073.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:23.341 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:23.342 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:19:23.342 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:23.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T14:19:23.994 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:24.125 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:24.125 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:Install 5 Packages 2024-09-21T14:19:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 3.8 M 2024-09-21T14:19:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 87 M 2024-09-21T14:19:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:24.347 INFO:teuthology.orchestra.run.smithi073.stdout:(1/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 106 kB/s | 16 kB 00:00 2024-09-21T14:19:24.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-21T14:19:24.372 INFO:teuthology.orchestra.run.smithi073.stdout:(2/5): ceph-grafana-dashboards-18.2.4-855.g19e2 135 kB/s | 24 kB 00:00 2024-09-21T14:19:24.395 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-21T14:19:24.414 INFO:teuthology.orchestra.run.smithi073.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 459 kB/s | 31 kB 00:00 2024-09-21T14:19:24.439 INFO:teuthology.orchestra.run.smithi073.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-21T14:19:24.556 INFO:teuthology.orchestra.run.smithi073.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 9.7 MB/s | 3.5 MB 00:00 2024-09-21T14:19:24.560 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:24.560 INFO:teuthology.orchestra.run.smithi073.stdout:Total 8.7 MB/s | 3.8 MB 00:00 2024-09-21T14:19:24.560 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:24.567 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:24.567 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:24.648 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:24.648 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:25.004 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-21T14:19:25.036 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-21T14:19:25.065 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-21T14:19:25.473 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-21T14:19:25.473 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-21T14:19:25.474 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-21T14:19:25.474 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-21T14:19:25.474 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:25.740 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:25.967 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T14:19:26.115 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-21T14:19:26.160 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-21T14:19:26.541 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:26.676 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T14:19:26.677 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout:Install 12 Packages 2024-09-21T14:19:26.678 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:26.679 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 38 M 2024-09-21T14:19:26.679 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 212 M 2024-09-21T14:19:26.679 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:26.973 INFO:teuthology.orchestra.run.smithi161.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 951 kB/s | 191 kB 00:00 2024-09-21T14:19:27.073 INFO:teuthology.orchestra.run.smithi161.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-21T14:19:27.098 INFO:teuthology.orchestra.run.smithi161.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 260 kB/s | 33 kB 00:00 2024-09-21T14:19:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-09-21T14:19:27.251 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-21T14:19:27.251 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-21T14:19:27.251 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-21T14:19:27.251 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-21T14:19:27.251 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-21T14:19:27.266 INFO:teuthology.orchestra.run.smithi161.stdout:(5/12): ceph-mgr-diskprediction-local-18.2.4-85 15 MB/s | 7.4 MB 00:00 2024-09-21T14:19:27.291 INFO:teuthology.orchestra.run.smithi161.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-21T14:19:27.391 INFO:teuthology.orchestra.run.smithi161.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T14:19:27.521 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:27.522 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:27.583 INFO:teuthology.orchestra.run.smithi161.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 16 MB/s | 4.9 MB 00:00 2024-09-21T14:19:27.659 INFO:teuthology.orchestra.run.smithi161.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-21T14:19:27.692 INFO:teuthology.orchestra.run.smithi161.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-09-21T14:19:27.759 INFO:teuthology.orchestra.run.smithi161.stdout:(11/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 4.4 MB/s | 3.0 MB 00:00 2024-09-21T14:19:27.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T14:19:28.034 INFO:teuthology.orchestra.run.smithi161.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-21T14:19:28.035 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:28.036 INFO:teuthology.orchestra.run.smithi161.stdout:Total 28 MB/s | 38 MB 00:01 2024-09-21T14:19:28.346 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:28.391 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:28.417 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:28.417 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:28.477 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:28.478 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T14:19:28.479 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout:Install 12 Packages 2024-09-21T14:19:28.480 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:28.481 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 38 M 2024-09-21T14:19:28.481 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 212 M 2024-09-21T14:19:28.481 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:28.781 INFO:teuthology.orchestra.run.smithi073.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-21T14:19:28.848 INFO:teuthology.orchestra.run.smithi073.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-21T14:19:28.944 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:28.944 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:29.048 INFO:teuthology.orchestra.run.smithi073.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-09-21T14:19:29.181 INFO:teuthology.orchestra.run.smithi073.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.4-85 14 MB/s | 7.4 MB 00:00 2024-09-21T14:19:29.365 INFO:teuthology.orchestra.run.smithi073.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 57 kB/s | 18 kB 00:00 2024-09-21T14:19:29.448 INFO:teuthology.orchestra.run.smithi073.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 144 kB/s | 38 kB 00:00 2024-09-21T14:19:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:29.757 INFO:teuthology.orchestra.run.smithi073.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.3 MB/s | 3.0 MB 00:00 2024-09-21T14:19:29.798 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T14:19:29.851 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T14:19:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T14:19:29.957 INFO:teuthology.orchestra.run.smithi073.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 481 kB/s | 245 kB 00:00 2024-09-21T14:19:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T14:19:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T14:19:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T14:19:30.266 INFO:teuthology.orchestra.run.smithi073.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 5.5 MB/s | 4.9 MB 00:00 2024-09-21T14:19:30.290 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T14:19:30.458 INFO:teuthology.orchestra.run.smithi073.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 501 kB/s | 250 kB 00:00 2024-09-21T14:19:30.725 INFO:teuthology.orchestra.run.smithi073.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.3 MB/s | 5.1 MB 00:00 2024-09-21T14:19:30.939 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T14:19:31.085 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T14:19:32.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T14:19:32.600 INFO:teuthology.orchestra.run.smithi073.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-09-21T14:19:32.601 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:32.601 INFO:teuthology.orchestra.run.smithi073.stdout:Total 9.3 MB/s | 38 MB 00:04 2024-09-21T14:19:32.961 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:32.987 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:32.987 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T14:19:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-21T14:19:33.487 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:33.487 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:34.244 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T14:19:34.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T14:19:34.496 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T14:19:34.541 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T14:19:34.727 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T14:19:34.757 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T14:19:34.834 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T14:19:35.477 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T14:19:35.593 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T14:19:35.961 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-21T14:19:35.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-21T14:19:35.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T14:19:35.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T14:19:35.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T14:19:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T14:19:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T14:19:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:36.350 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:36.751 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-rook 2024-09-21T14:19:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T14:19:37.344 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:37.476 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:37.478 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:37.478 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:37.478 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:37.478 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:37.478 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T14:19:37.479 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout:Install 13 Packages 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:37.480 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 2.2 M 2024-09-21T14:19:37.481 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 26 M 2024-09-21T14:19:37.481 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:37.789 INFO:teuthology.orchestra.run.smithi161.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 247 kB/s | 26 kB 00:00 2024-09-21T14:19:37.814 INFO:teuthology.orchestra.run.smithi161.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 148 kB/s | 19 kB 00:00 2024-09-21T14:19:37.864 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T14:19:37.873 INFO:teuthology.orchestra.run.smithi161.stdout:(3/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 262 kB/s | 49 kB 00:00 2024-09-21T14:19:37.889 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-21T14:19:37.898 INFO:teuthology.orchestra.run.smithi161.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-21T14:19:37.932 INFO:teuthology.orchestra.run.smithi161.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T14:19:37.974 INFO:teuthology.orchestra.run.smithi161.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 772 kB/s | 32 kB 00:00 2024-09-21T14:19:37.999 INFO:teuthology.orchestra.run.smithi161.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 533 kB/s | 54 kB 00:00 2024-09-21T14:19:38.024 INFO:teuthology.orchestra.run.smithi161.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-21T14:19:38.075 INFO:teuthology.orchestra.run.smithi161.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-21T14:19:38.125 INFO:teuthology.orchestra.run.smithi161.stdout:(10/13): python3-kubernetes-26.1.0-2.el9.noarch 11 MB/s | 1.0 MB 00:00 2024-09-21T14:19:38.150 INFO:teuthology.orchestra.run.smithi161.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-21T14:19:38.175 INFO:teuthology.orchestra.run.smithi161.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 599 kB/s | 59 kB 00:00 2024-09-21T14:19:38.201 INFO:teuthology.orchestra.run.smithi161.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-21T14:19:38.203 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:38.203 INFO:teuthology.orchestra.run.smithi161.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-21T14:19:38.268 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:38.283 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:38.284 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:38.374 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:38.375 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:38.891 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:39.063 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T14:19:39.110 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T14:19:39.174 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T14:19:39.219 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T14:19:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T14:19:39.362 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T14:19:39.474 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T14:19:39.528 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T14:19:40.250 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T14:19:40.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T14:19:40.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T14:19:40.391 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T14:19:40.413 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-21T14:19:40.489 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-21T14:19:40.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-21T14:19:40.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T14:19:40.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T14:19:40.888 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:40.889 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T14:19:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T14:19:41.210 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T14:19:41.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-rook 2024-09-21T14:19:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T14:19:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:41.548 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T14:19:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:41.550 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:41.826 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T14:19:41.884 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:11 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:42.025 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:42.026 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:42.026 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T14:19:42.027 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout:Install 13 Packages 2024-09-21T14:19:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:42.029 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 2.2 M 2024-09-21T14:19:42.029 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 26 M 2024-09-21T14:19:42.029 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:42.418 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:14 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:42.551 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:42.551 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:42.551 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:42.551 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:42.552 INFO:teuthology.orchestra.run.smithi161.stdout:Install 6 Packages 2024-09-21T14:19:42.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:42.553 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 7.0 M 2024-09-21T14:19:42.553 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 33 M 2024-09-21T14:19:42.553 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:42.606 INFO:teuthology.orchestra.run.smithi073.stdout:(1/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 306 kB/s | 49 kB 00:00 2024-09-21T14:19:42.740 INFO:teuthology.orchestra.run.smithi073.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 65 kB/s | 19 kB 00:00 2024-09-21T14:19:42.774 INFO:teuthology.orchestra.run.smithi073.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 78 kB/s | 26 kB 00:00 2024-09-21T14:19:43.116 INFO:teuthology.orchestra.run.smithi073.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-21T14:19:43.166 INFO:teuthology.orchestra.run.smithi073.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 397 kB/s | 222 kB 00:00 2024-09-21T14:19:43.208 INFO:teuthology.orchestra.run.smithi073.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-21T14:19:43.234 INFO:teuthology.orchestra.run.smithi073.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 458 kB/s | 54 kB 00:00 2024-09-21T14:19:43.259 INFO:teuthology.orchestra.run.smithi073.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 282 kB/s | 14 kB 00:00 2024-09-21T14:19:43.292 INFO:teuthology.orchestra.run.smithi073.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 538 kB/s | 279 kB 00:00 2024-09-21T14:19:43.334 INFO:teuthology.orchestra.run.smithi073.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-21T14:19:43.360 INFO:teuthology.orchestra.run.smithi073.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-21T14:19:43.378 INFO:teuthology.orchestra.run.smithi161.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-21T14:19:43.385 INFO:teuthology.orchestra.run.smithi073.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 653 kB/s | 59 kB 00:00 2024-09-21T14:19:43.403 INFO:teuthology.orchestra.run.smithi161.stdout:(2/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 606 kB/s | 139 kB 00:00 2024-09-21T14:19:43.427 INFO:teuthology.orchestra.run.smithi073.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.3 MB/s | 1.0 MB 00:00 2024-09-21T14:19:43.429 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:43.429 INFO:teuthology.orchestra.run.smithi073.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-09-21T14:19:43.462 INFO:teuthology.orchestra.run.smithi161.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-21T14:19:43.479 INFO:teuthology.orchestra.run.smithi161.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 765 kB/s | 58 kB 00:00 2024-09-21T14:19:43.493 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:43.504 INFO:teuthology.orchestra.run.smithi161.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-21T14:19:43.508 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:43.508 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:43.596 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:43.605 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:44.106 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:44.284 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T14:19:44.313 INFO:teuthology.orchestra.run.smithi161.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-21T14:19:44.315 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:44.316 INFO:teuthology.orchestra.run.smithi161.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-09-21T14:19:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T14:19:44.386 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:44.398 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:44.398 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:44.400 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T14:19:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T14:19:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T14:19:44.497 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:44.497 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:44.580 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T14:19:44.690 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T14:19:44.747 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T14:19:44.813 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T14:19:45.039 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T14:19:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T14:19:45.528 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T14:19:45.561 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T14:19:45.563 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T14:19:45.609 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T14:19:45.631 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-21T14:19:45.650 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T14:19:45.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T14:19:45.743 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T14:19:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T14:19:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T14:19:46.557 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T14:19:46.557 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T14:19:46.557 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T14:19:46.557 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T14:19:46.788 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:46.789 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:46.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T14:19:46.893 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:46.893 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:46.893 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:46.893 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:46.894 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:47.052 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-fuse 2024-09-21T14:19:47.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T14:19:47.639 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:47.736 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:17 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:47.772 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout:Install 2 Packages 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:47.773 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 932 k 2024-09-21T14:19:47.774 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 2.7 M 2024-09-21T14:19:47.774 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:47.868 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:47.869 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:47.869 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:47.869 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout:Install 6 Packages 2024-09-21T14:19:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:47.871 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 7.0 M 2024-09-21T14:19:47.871 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 33 M 2024-09-21T14:19:47.871 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:48.014 INFO:teuthology.orchestra.run.smithi161.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 698 kB/s | 80 kB 00:00 2024-09-21T14:19:48.189 INFO:teuthology.orchestra.run.smithi161.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 2.9 MB/s | 852 kB 00:00 2024-09-21T14:19:48.190 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:48.190 INFO:teuthology.orchestra.run.smithi161.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-21T14:19:48.202 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:48.224 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:48.224 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:48.279 INFO:teuthology.orchestra.run.smithi073.stdout:(1/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 640 kB/s | 139 kB 00:00 2024-09-21T14:19:48.303 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:48.303 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:48.338 INFO:teuthology.orchestra.run.smithi073.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-21T14:19:48.388 INFO:teuthology.orchestra.run.smithi073.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T14:19:48.422 INFO:teuthology.orchestra.run.smithi073.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.3 MB/s | 74 kB 00:00 2024-09-21T14:19:48.495 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:48.629 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T14:19:48.705 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-21T14:19:48.906 INFO:teuthology.orchestra.run.smithi073.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 294 kB/s | 248 kB 00:00 2024-09-21T14:19:49.272 INFO:teuthology.orchestra.run.smithi073.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-21T14:19:49.275 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:49.275 INFO:teuthology.orchestra.run.smithi073.stdout:Total 5.0 MB/s | 7.0 MB 00:01 2024-09-21T14:19:49.348 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:49.360 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:49.360 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-21T14:19:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-21T14:19:49.460 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:49.461 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:49.729 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T14:19:49.730 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:49.730 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:49.730 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T14:19:49.730 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:49.730 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:49.947 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T14:19:49.994 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-volume 2024-09-21T14:19:49.999 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T14:19:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T14:19:50.596 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:22 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:50.611 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T14:19:50.672 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T14:19:50.695 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-21T14:19:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:50.733 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:Install 2 Packages 2024-09-21T14:19:50.734 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 342 k 2024-09-21T14:19:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 1.5 M 2024-09-21T14:19:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:50.903 INFO:teuthology.orchestra.run.smithi161.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 636 kB/s | 77 kB 00:00 2024-09-21T14:19:51.029 INFO:teuthology.orchestra.run.smithi161.stdout:(2/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.0 MB/s | 264 kB 00:00 2024-09-21T14:19:51.029 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:51.029 INFO:teuthology.orchestra.run.smithi161.stdout:Total 1.1 MB/s | 342 kB 00:00 2024-09-21T14:19:51.041 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:51.050 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:51.050 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:51.087 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:51.088 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:51.231 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:51.420 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-21T14:19:51.464 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-21T14:19:51.481 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:51.482 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T14:19:51.482 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T14:19:51.752 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T14:19:51.753 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:51.753 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:52.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-fuse 2024-09-21T14:19:52.211 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-21T14:19:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-21T14:19:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:52.502 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:52.502 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T14:19:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:52.503 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:52.617 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:22 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:52.753 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T14:19:52.754 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:Install 2 Packages 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 932 k 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 2.7 M 2024-09-21T14:19:52.755 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:52.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librados-devel 2024-09-21T14:19:53.033 INFO:teuthology.orchestra.run.smithi073.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 537 kB/s | 80 kB 00:00 2024-09-21T14:19:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 2.9 MB/s | 852 kB 00:00 2024-09-21T14:19:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-21T14:19:53.178 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:53.200 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:53.200 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:53.394 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:53.463 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:53.529 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:53.529 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2024-09-21T14:19:53.529 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:53.529 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 126 k 2024-09-21T14:19:53.530 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 456 k 2024-09-21T14:19:53.531 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:53.598 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T14:19:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-21T14:19:53.734 INFO:teuthology.orchestra.run.smithi161.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 623 kB/s | 126 kB 00:00 2024-09-21T14:19:53.734 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:53.735 INFO:teuthology.orchestra.run.smithi161.stdout:Total 618 kB/s | 126 kB 00:00 2024-09-21T14:19:53.735 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:53.740 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:53.740 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:53.805 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:53.806 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:53.954 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:54.056 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:54.350 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-21T14:19:54.350 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-21T14:19:54.526 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:54.635 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T14:19:54.636 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:54.636 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:54.636 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T14:19:54.636 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:54.636 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:54.770 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:54.946 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-volume 2024-09-21T14:19:54.977 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs2 2024-09-21T14:19:55.543 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:55.570 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:55.661 INFO:teuthology.orchestra.run.smithi161.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:55.679 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:55.679 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:55.679 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout:Install 2 Packages 2024-09-21T14:19:55.680 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:55.681 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 342 k 2024-09-21T14:19:55.681 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 1.5 M 2024-09-21T14:19:55.681 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:55.705 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:55.707 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:55.707 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:55.777 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs-devel 2024-09-21T14:19:55.905 INFO:teuthology.orchestra.run.smithi073.stdout:(1/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.3 MB/s | 264 kB 00:00 2024-09-21T14:19:56.172 INFO:teuthology.orchestra.run.smithi073.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 165 kB/s | 77 kB 00:00 2024-09-21T14:19:56.172 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:56.172 INFO:teuthology.orchestra.run.smithi073.stdout:Total 694 kB/s | 342 kB 00:00 2024-09-21T14:19:56.184 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:56.193 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:56.193 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:56.230 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:56.231 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:56.362 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:56.371 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:56.495 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:56.496 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:56.496 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2024-09-21T14:19:56.496 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:56.496 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 31 k 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 139 k 2024-09-21T14:19:56.497 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:19:56.545 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-21T14:19:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-21T14:19:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-21T14:19:56.606 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:19:56.606 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T14:19:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:56.653 INFO:teuthology.orchestra.run.smithi161.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 199 kB/s | 31 kB 00:00 2024-09-21T14:19:56.654 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:56.654 INFO:teuthology.orchestra.run.smithi161.stdout:Total 197 kB/s | 31 kB 00:00 2024-09-21T14:19:56.654 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:19:56.658 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:19:56.658 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:19:56.682 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:19:56.682 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:19:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:19:56.884 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:57.258 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:57.314 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:19:57.511 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:57.620 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:57.694 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librados2 2024-09-21T14:19:57.912 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librados-devel 2024-09-21T14:19:58.280 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:58.369 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:58.412 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:58.413 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:58.414 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:58.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librbd1 2024-09-21T14:19:58.512 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:19:58.644 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:19:58.645 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:19:58.646 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:58.646 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 126 k 2024-09-21T14:19:58.646 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 456 k 2024-09-21T14:19:58.646 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:19:58.852 INFO:teuthology.orchestra.run.smithi073.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 615 kB/s | 126 kB 00:00 2024-09-21T14:19:58.853 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:19:58.853 INFO:teuthology.orchestra.run.smithi073.stdout:Total 610 kB/s | 126 kB 00:00 2024-09-21T14:19:58.853 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:19:58.858 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:19:58.859 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:19:58.921 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:19:58.921 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:19:59.068 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:19:59.075 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:59.165 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:59.174 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:59.209 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:59.210 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:59.210 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:19:59.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rados 2024-09-21T14:19:59.636 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:59.849 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:19:59.893 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:19:59.938 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:19:59.981 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:19:59.981 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:19:59.982 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:00.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rgw 2024-09-21T14:20:00.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install libcephfs2 2024-09-21T14:20:00.633 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:32 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:00.724 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:00.751 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:00.766 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:00.768 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:20:00.768 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:00.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-cephfs 2024-09-21T14:20:00.842 INFO:teuthology.orchestra.run.smithi073.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:00.886 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:00.888 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:00.888 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:00.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install libcephfs-devel 2024-09-21T14:20:01.417 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:01.508 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:01.551 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:01.551 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:20:01.552 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:01.566 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:01.614 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rbd 2024-09-21T14:20:01.711 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:01.711 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-21T14:20:01.711 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:01.711 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 31 k 2024-09-21T14:20:01.712 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 139 k 2024-09-21T14:20:01.713 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:20:01.873 INFO:teuthology.orchestra.run.smithi073.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 194 kB/s | 31 kB 00:00 2024-09-21T14:20:01.873 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:01.874 INFO:teuthology.orchestra.run.smithi073.stdout:Total 192 kB/s | 31 kB 00:00 2024-09-21T14:20:01.874 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:20:01.877 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:20:01.877 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:20:01.902 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:20:01.902 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:20:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:20:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:02.198 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:02.289 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:02.332 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:02.333 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-09-21T14:20:02.334 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:02.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-fuse 2024-09-21T14:20:02.476 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:02.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:02.708 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:02.708 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:20:02.708 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:02.708 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:02.708 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:02.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librados2 2024-09-21T14:20:02.983 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:03.116 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:03.116 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:03.116 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:03.118 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 87 k 2024-09-21T14:20:03.118 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 227 k 2024-09-21T14:20:03.118 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:20:03.323 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 422 kB/s | 87 kB 00:00 2024-09-21T14:20:03.324 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:03.324 INFO:teuthology.orchestra.run.smithi161.stdout:Total 419 kB/s | 87 kB 00:00 2024-09-21T14:20:03.324 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:20:03.331 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:20:03.331 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:20:03.397 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:20:03.398 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:20:03.498 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:20:03.587 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:03.630 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:03.632 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:03.632 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:03.654 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:03.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librbd1 2024-09-21T14:20:04.021 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:04.278 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:04.294 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:04.366 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:04.408 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:04.409 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:04.409 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:04.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rados 2024-09-21T14:20:04.558 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-mirror 2024-09-21T14:20:05.055 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:05.145 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:05.189 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:05.190 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:05.190 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:05.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rgw 2024-09-21T14:20:05.281 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:20:05.282 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:05.283 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:20:05.283 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:05.283 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 3.0 M 2024-09-21T14:20:05.283 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 12 M 2024-09-21T14:20:05.283 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:20:05.610 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-09-21T14:20:05.610 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:05.611 INFO:teuthology.orchestra.run.smithi161.stdout:Total 9.1 MB/s | 3.0 MB 00:00 2024-09-21T14:20:05.611 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:20:05.620 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:20:05.620 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:20:05.700 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:20:05.700 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:20:05.842 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:05.931 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:05.974 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:05.975 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:05.975 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:06.046 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-cephfs 2024-09-21T14:20:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:20:06.177 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:06.202 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:06.203 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:20:06.203 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T14:20:06.203 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T14:20:06.203 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T14:20:06.203 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:06.629 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:06.722 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:06.765 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:06.766 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:06.766 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:06.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rbd 2024-09-21T14:20:07.326 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:07.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:07.326 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:20:07.327 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:07.327 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:07.327 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:07.414 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:07.504 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-21T14:20:07.547 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:07.548 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-21T14:20:07.548 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:07.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-nbd 2024-09-21T14:20:07.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-fuse 2024-09-21T14:20:08.173 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 02:18:28 PM UTC. 2024-09-21T14:20:08.194 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:08.306 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T14:20:08.306 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:08.307 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-09-21T14:20:08.307 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:08.309 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T14:20:08.309 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-21T14:20:08.309 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:08.309 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T14:20:08.309 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-09-21T14:20:08.310 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2024-09-21T14:20:08.310 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:08.310 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 172 k 2024-09-21T14:20:08.310 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 494 k 2024-09-21T14:20:08.310 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T14:20:08.325 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:08.325 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:08.325 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:20:08.326 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:08.327 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 87 k 2024-09-21T14:20:08.327 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 227 k 2024-09-21T14:20:08.327 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:20:08.518 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 455 kB/s | 87 kB 00:00 2024-09-21T14:20:08.518 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:08.518 INFO:teuthology.orchestra.run.smithi073.stdout:Total 452 kB/s | 87 kB 00:00 2024-09-21T14:20:08.518 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:20:08.525 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:20:08.525 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:20:08.529 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 776 kB/s | 172 kB 00:00 2024-09-21T14:20:08.530 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:08.530 INFO:teuthology.orchestra.run.smithi161.stdout:Total 770 kB/s | 172 kB 00:00 2024-09-21T14:20:08.531 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T14:20:08.538 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T14:20:08.538 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T14:20:08.587 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:20:08.587 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:20:08.602 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T14:20:08.602 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T14:20:08.741 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:20:08.757 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T14:20:08.847 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:08.860 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:09.345 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:09.569 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T14:20:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:09.569 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T14:20:09.664 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:09.664 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:09.664 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:20:09.665 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:09.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:09.665 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:09.831 DEBUG:teuthology.parallel:result is None 2024-09-21T14:20:09.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-mirror 2024-09-21T14:20:10.527 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:20:10.665 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 3.0 M 2024-09-21T14:20:10.666 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 12 M 2024-09-21T14:20:10.667 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:20:11.021 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 8.4 MB/s | 3.0 MB 00:00 2024-09-21T14:20:11.021 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:11.022 INFO:teuthology.orchestra.run.smithi073.stdout:Total 8.4 MB/s | 3.0 MB 00:00 2024-09-21T14:20:11.022 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:20:11.031 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:20:11.031 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:20:11.110 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:20:11.110 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:20:11.526 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:20:11.588 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T14:20:11.613 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:12.707 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:12.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-nbd 2024-09-21T14:20:13.533 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 02:18:30 PM UTC. 2024-09-21T14:20:13.668 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-21T14:20:13.669 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:13.669 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-21T14:20:13.669 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 172 k 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 494 k 2024-09-21T14:20:13.670 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-21T14:20:13.883 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 807 kB/s | 172 kB 00:00 2024-09-21T14:20:13.884 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-21T14:20:13.884 INFO:teuthology.orchestra.run.smithi073.stdout:Total 801 kB/s | 172 kB 00:00 2024-09-21T14:20:13.884 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-21T14:20:13.892 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-21T14:20:13.892 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-21T14:20:13.956 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-21T14:20:13.956 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-21T14:20:14.114 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-21T14:20:14.222 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:14.950 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-21T14:20:15.196 DEBUG:teuthology.parallel:result is None 2024-09-21T14:20:15.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:20:15.359 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T14:20:15.381 INFO:teuthology.orchestra.run.smithi073.stdout:18.2.4-855.g19e2a2be.el9 2024-09-21T14:20:15.381 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-21T14:20:15.381 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-21T14:20:15.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:20:15.531 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T14:20:15.557 INFO:teuthology.orchestra.run.smithi161.stdout:18.2.4-855.g19e2a2be.el9 2024-09-21T14:20:15.557 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-21T14:20:15.557 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-21T14:20:15.559 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T14:20:15.559 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:15.559 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T14:20:15.586 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:15.586 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T14:20:15.628 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T14:20:15.628 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:15.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T14:20:15.654 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T14:20:15.719 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:15.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T14:20:15.748 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T14:20:15.816 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T14:20:15.817 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:15.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T14:20:15.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T14:20:15.913 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:15.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T14:20:15.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T14:20:16.009 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T14:20:16.009 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:16.009 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T14:20:16.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T14:20:16.098 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:16.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T14:20:16.125 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T14:20:16.192 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T14:20:16.264 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T14:20:16.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-09-21T14:20:16.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2024-09-21T14:20:16.292 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T14:20:16.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T14:20:16.331 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T14:20:16.357 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T14:20:16.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T14:20:16.395 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T14:20:16.441 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T14:20:16.441 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T14:20:16.441 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909929', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4623, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'sleep_before_teardown': 0, 'subset': '80/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7', 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATemBgD1awZxYY4oEDXshWP0benAYdv4h2PCgE5fnHD1BUfB7qTIspOXfLh5KERlUcigsuKutrVDnyBMJdUK2w=', 'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHhDk/uCjD4mo2q5T0Px8nYodV4mxydcWs3fA43NZMkR51jG1eGqi17tpDxEkqrrHAAhDJ+PgaeALI9RU703yM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_22:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T14:20:16.442 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T14:20:16.460 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T14:20:16.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T14:20:16.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T14:20:16.545 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:16.545 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T14:20:16.594 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T14:20:16.594 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-09-21T14:20:16.649 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T14:20:16.649 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:16.649 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-21 14:20:11.869296141 +0000 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-21 14:17:47.321642477 +0000 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-21 14:17:47.321642477 +0000 2024-09-21T14:20:16.650 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-21 14:17:47.321642477 +0000 2024-09-21T14:20:16.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T14:20:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-21T14:20:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-21T14:20:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000180311 s, 2.8 MB/s 2024-09-21T14:20:16.714 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T14:20:16.770 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-09-21T14:20:16.825 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T14:20:16.825 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:16.825 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-21 14:20:11.869296141 +0000 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-21 14:17:47.646635116 +0000 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-21 14:17:47.646635116 +0000 2024-09-21T14:20:16.826 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-21 14:17:47.646635116 +0000 2024-09-21T14:20:16.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T14:20:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-21T14:20:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-21T14:20:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000186519 s, 2.7 MB/s 2024-09-21T14:20:16.890 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T14:20:16.946 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-21 14:20:11.870296119 +0000 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-21 14:17:47.963627936 +0000 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-21 14:17:47.963627936 +0000 2024-09-21T14:20:17.002 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-21 14:17:47.963627936 +0000 2024-09-21T14:20:17.003 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T14:20:17.065 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-21T14:20:17.065 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-21T14:20:17.065 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.00016675 s, 3.1 MB/s 2024-09-21T14:20:17.066 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T14:20:17.122 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-21 14:20:11.870296119 +0000 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-21 14:17:48.279620779 +0000 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-21 14:17:48.279620779 +0000 2024-09-21T14:20:17.177 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-21 14:17:48.279620779 +0000 2024-09-21T14:20:17.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T14:20:17.239 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-21T14:20:17.240 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-21T14:20:17.240 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000194109 s, 2.6 MB/s 2024-09-21T14:20:17.241 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T14:20:17.296 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:17.297 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:17.297 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T14:20:17.317 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T14:20:17.317 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-09-21T14:20:17.375 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T14:20:17.375 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.375 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 14:20:06.468427378 +0000 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 14:17:47.383007795 +0000 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 14:17:47.383007795 +0000 2024-09-21T14:20:17.376 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 14:17:47.383007795 +0000 2024-09-21T14:20:17.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T14:20:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T14:20:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T14:20:17.443 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000289363 s, 1.8 MB/s 2024-09-21T14:20:17.444 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T14:20:17.504 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-09-21T14:20:17.561 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T14:20:17.561 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.561 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T14:20:17.561 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.561 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.562 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 14:20:06.468427378 +0000 2024-09-21T14:20:17.562 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 14:17:47.702001490 +0000 2024-09-21T14:20:17.562 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 14:17:47.702001490 +0000 2024-09-21T14:20:17.562 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 14:17:47.702001490 +0000 2024-09-21T14:20:17.562 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T14:20:17.627 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T14:20:17.627 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T14:20:17.627 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000231796 s, 2.2 MB/s 2024-09-21T14:20:17.629 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T14:20:17.686 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 14:20:06.469427358 +0000 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 14:17:48.022995146 +0000 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 14:17:48.022995146 +0000 2024-09-21T14:20:17.743 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 14:17:48.022995146 +0000 2024-09-21T14:20:17.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T14:20:17.807 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T14:20:17.807 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T14:20:17.807 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000271271 s, 1.9 MB/s 2024-09-21T14:20:17.809 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T14:20:17.865 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-09-21T14:20:17.921 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 14:20:06.469427358 +0000 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 14:17:48.335988959 +0000 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 14:17:48.335988959 +0000 2024-09-21T14:20:17.922 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 14:17:48.335988959 +0000 2024-09-21T14:20:17.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T14:20:17.986 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T14:20:17.986 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T14:20:17.986 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000219198 s, 2.3 MB/s 2024-09-21T14:20:17.987 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T14:20:18.043 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:18.043 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T14:20:18.043 INFO:tasks.ceph:Generating config... 2024-09-21T14:20:18.044 INFO:tasks.ceph:[client] rbd cache = True 2024-09-21T14:20:18.044 INFO:tasks.ceph:[client] rbd cache max dirty = 0 2024-09-21T14:20:18.045 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T14:20:18.045 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T14:20:18.045 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T14:20:18.045 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T14:20:18.045 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T14:20:18.045 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T14:20:18.045 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T14:20:18.045 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T14:20:18.045 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T14:20:18.046 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T14:20:18.047 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T14:20:18.047 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T14:20:18.102 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T14:20:18.105 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T14:20:18.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:20:18.174 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.c', '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]'), ('mon.b', '172.21.15.161')] 2024-09-21T14:20:18.175 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],172.21.15.161', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache max dirty': 0}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T14:20:18.175 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:18.176 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T14:20:18.231 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --addv c '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]' --add b 172.21.15.161 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid da87574b-7df0-407c-8805-347c49d2d055 2024-09-21T14:20:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = pacific 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:fsid da87574b-7df0-407c-8805-347c49d2d055 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-09-21T14:20:18.317409+0000 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-21T14:20:18.317409+0000 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 16 (pacific) 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 1 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] mon.c 2024-09-21T14:20:18.317 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T14:20:18.318 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T14:20:18.333 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID da87574b-7df0-407c-8805-347c49d2d055... 2024-09-21T14:20:18.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T14:20:18.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T14:20:18.415 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-09-21T14:20:18.415 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-09-21T14:20:18.415 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T14:20:18.415 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-09-21T14:20:18.415 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-09-21T14:20:18.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T14:20:18.417 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-09-21T14:20:18.418 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],172.21.15.161" 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: mon client directed command retry = 5 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on pool no app = False 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: ms inject socket failures = 5000 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = da87574b-7df0-407c-8805-347c49d2d055 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.419 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.420 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore compression algorithm = snappy 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore compression mode = aggressive 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-21T14:20:18.421 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: osd blocked scrub grace period = 3600 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: osd max markdown count = 1000 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: osd shutdown pgref assert = True 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-21T14:20:18.422 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:20:18.423 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-09-21T14:20:18.424 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: rbd cache = True 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout: rbd cache max dirty = 0 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-09-21T14:20:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2024-09-21T14:20:18.426 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2024-09-21T14:20:18.427 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.428 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],172.21.15.161" 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon client directed command retry = 5 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on pool no app = False 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: ms inject socket failures = 5000 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = da87574b-7df0-407c-8805-347c49d2d055 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2024-09-21T14:20:18.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2024-09-21T14:20:18.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore compression algorithm = snappy 2024-09-21T14:20:18.431 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore compression mode = aggressive 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 1/20 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 1/20 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 4/10 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd blocked scrub grace period = 3600 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 0.95 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd max markdown count = 1000 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = bluestore 2024-09-21T14:20:18.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd shutdown pgref assert = True 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2024-09-21T14:20:18.433 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T14:20:18.434 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: rbd cache = True 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout: rbd cache max dirty = 0 2024-09-21T14:20:18.435 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2024-09-21T14:20:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2024-09-21T14:20:18.436 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2024-09-21T14:20:18.437 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T14:20:18.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T14:20:18.530 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T14:20:18.531 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:18.531 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T14:20:18.546 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:18.546 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T14:20:18.601 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:20:18.601 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:18.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T14:20:18.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:20:18.675 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:18.675 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:18.730 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:20:18.730 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:18.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T14:20:18.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:20:18.766 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:18.766 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:18.822 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T14:20:18.822 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T14:20:18.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T14:20:18.874 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T14:20:18.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T14:20:18.935 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T14:20:18.938 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T14:20:18.938 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T14:20:18.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T14:20:18.978 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T14:20:18.991 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T14:20:19.031 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T14:20:19.044 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T14:20:19.044 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T14:20:19.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T14:20:19.068 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:19.068 INFO:tasks.ceph:role: osd.0 2024-09-21T14:20:19.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:20:19.069 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T14:20:19.140 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:19.140 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:19.141 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:19.649 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-21T14:20:19.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-21T14:20:19.651 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T14:20:19.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T14:20:19.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T14:20:19.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:19.877 INFO:tasks.ceph:role: osd.1 2024-09-21T14:20:19.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:20:19.878 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T14:20:19.950 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:19.950 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:19.951 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:20.415 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-21T14:20:20.417 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-21T14:20:20.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T14:20:20.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T14:20:20.572 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T14:20:20.637 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:20.637 INFO:tasks.ceph:role: osd.2 2024-09-21T14:20:20.637 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:20:20.637 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:20.709 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:21.212 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-21T14:20:21.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-21T14:20:21.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T14:20:21.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T14:20:21.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T14:20:21.443 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:21.443 INFO:tasks.ceph:role: osd.3 2024-09-21T14:20:21.444 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:20:21.444 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T14:20:21.517 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:21.518 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:22.023 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-21T14:20:22.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-21T14:20:22.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T14:20:22.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T14:20:22.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.292+0000 7f4936016640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T14:20:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.293+0000 7f4936016640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T14:20:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.293+0000 7f4936016640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:20:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.293+0000 7f4936016640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:20:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.293+0000 7f4936016640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:20:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:22.293+0000 7f4936016640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T14:20:25.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T14:20:25.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:25.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.267+0000 7fae109c3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T14:20:25.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.268+0000 7fae109c3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T14:20:25.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.268+0000 7fae109c3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:20:25.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.268+0000 7fae109c3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:20:25.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.268+0000 7fae109c3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:20:25.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:25.268+0000 7fae109c3640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T14:20:28.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T14:20:28.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.249+0000 7f034b00f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.249+0000 7f034b00f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.249+0000 7f034b00f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.250+0000 7f034b00f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.250+0000 7f034b00f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:20:28.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:28.250+0000 7f034b00f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T14:20:31.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T14:20:31.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.209+0000 7f80212d8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.209+0000 7f80212d8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.209+0000 7f80212d8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.209+0000 7f80212d8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.209+0000 7f80212d8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:20:31.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-21T14:20:31.210+0000 7f80212d8640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T14:20:34.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T14:20:34.083 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T14:20:34.114 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:34.114 INFO:tasks.ceph:role: osd.4 2024-09-21T14:20:34.114 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:20:34.114 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T14:20:34.190 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:34.191 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:34.700 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-09-21T14:20:34.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-09-21T14:20:34.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T14:20:34.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T14:20:34.854 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T14:20:34.921 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:34.921 INFO:tasks.ceph:role: osd.5 2024-09-21T14:20:34.922 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:20:34.922 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T14:20:34.995 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:34.995 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:34.995 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:34.995 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:34.995 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:34.996 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:34.996 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:34.996 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:34.997 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:34.997 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:35.460 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-09-21T14:20:35.462 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-09-21T14:20:35.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T14:20:35.554 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T14:20:35.586 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T14:20:35.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:35.651 INFO:tasks.ceph:role: osd.6 2024-09-21T14:20:35.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:20:35.652 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:35.725 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:36.211 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-09-21T14:20:36.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-09-21T14:20:36.213 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T14:20:36.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T14:20:36.397 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T14:20:36.463 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:20:36.463 INFO:tasks.ceph:role: osd.7 2024-09-21T14:20:36.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:20:36.464 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T14:20:36.534 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:20:36.535 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:20:37.060 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-09-21T14:20:37.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-09-21T14:20:37.063 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T14:20:37.203 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T14:20:37.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.332+0000 7fcfc8920640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.333+0000 7fcfc8920640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.333+0000 7fcfc8920640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.333+0000 7fcfc8920640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.333+0000 7fcfc8920640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:20:37.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:37.333+0000 7fcfc8920640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T14:20:40.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T14:20:40.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:40.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.297+0000 7f829b414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T14:20:40.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.297+0000 7f829b414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T14:20:40.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.297+0000 7f829b414640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:20:40.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.297+0000 7f829b414640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:20:40.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.297+0000 7f829b414640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:20:40.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:40.298+0000 7f829b414640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T14:20:43.146 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T14:20:43.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:43.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.259+0000 7f16aa0b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T14:20:43.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.259+0000 7f16aa0b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T14:20:43.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.259+0000 7f16aa0b0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:20:43.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.259+0000 7f16aa0b0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:20:43.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.259+0000 7f16aa0b0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:20:43.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:43.260+0000 7f16aa0b0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T14:20:46.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T14:20:46.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:46.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.216+0000 7f1eac545640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T14:20:46.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.217+0000 7f1eac545640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T14:20:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.217+0000 7f1eac545640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:20:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.217+0000 7f1eac545640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:20:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.217+0000 7f1eac545640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:20:46.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:46.217+0000 7f1eac545640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T14:20:49.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T14:20:49.104 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T14:20:49.104 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T14:20:49.134 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T14:20:49.203 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.203 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T14:20:49.271 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T14:20:49.337 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T14:20:49.404 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.404 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T14:20:49.433 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.433 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T14:20:49.503 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T14:20:49.572 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T14:20:49.640 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T14:20:49.707 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:49.708 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T14:20:49.725 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:49.725 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T14:20:49.764 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T14:20:49.764 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T14:20:49.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAi1u5mtEAvNBAAHMKvtTaFnZQXUayo98iqPw== 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAm1u5mvbyFERAAiUMzVnRZ0qQiHSM1Zkpu8g== 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-09-21T14:20:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAp1u5mkG4XEBAAVWCvU5BmVZHQsWumJFk5wQ== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAs1u5meJrwDhAAqjpcGUFbzJ/sUoBKIgewLQ== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAv1u5mnLGMDBAAuIxVhz4AX7mXr65G4t6sXA== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAi1u5mkwfQNxAAEhABy7Am8dXOaDB0k+lYpA== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA11u5muIDhExAA2bHr0FVzkhMF2ZvWzfVpQg== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA41u5mdF7HERAAPThMkOLKw7rurGph03fnXg== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA71u5mC3yCDxAA7jMkecnA2oW6lajZM08BOA== 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-09-21T14:20:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA+1u5mSn77DBAAkwUOZH6Lj5fVG5Q3KVU20g== 2024-09-21T14:20:49.810 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-09-21T14:20:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAi1u5mRRZkOhAAG0UkXzzYnVyJgDi8qH1Y0A== 2024-09-21T14:20:49.810 INFO:teuthology.orchestra.run.smithi073.stdout:[client.1] 2024-09-21T14:20:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAj1u5mlMHyARAA1vmPgT0dWCdb8E9G9o4r8g== 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAi1u5mtEAvNBAAHMKvtTaFnZQXUayo98iqPw== 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAm1u5mvbyFERAAiUMzVnRZ0qQiHSM1Zkpu8g== 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAp1u5mkG4XEBAAVWCvU5BmVZHQsWumJFk5wQ== 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAs1u5meJrwDhAAqjpcGUFbzJ/sUoBKIgewLQ== 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2024-09-21T14:20:49.829 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAv1u5mnLGMDBAAuIxVhz4AX7mXr65G4t6sXA== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAi1u5mkwfQNxAAEhABy7Am8dXOaDB0k+lYpA== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQA11u5muIDhExAA2bHr0FVzkhMF2ZvWzfVpQg== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQA41u5mdF7HERAAPThMkOLKw7rurGph03fnXg== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQA71u5mC3yCDxAA7jMkecnA2oW6lajZM08BOA== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQA+1u5mSn77DBAAkwUOZH6Lj5fVG5Q3KVU20g== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAi1u5mRRZkOhAAG0UkXzzYnVyJgDi8qH1Y0A== 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-09-21T14:20:49.830 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAj1u5mlMHyARAA1vmPgT0dWCdb8E9G9o4r8g== 2024-09-21T14:20:49.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:20:49.854 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:20:49.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:49.926 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:49.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:49.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.029 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.033 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.136 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:20:50.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:20:50.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.191 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.243 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.294 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.346 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:20:50.396 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:20:50.399 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:20:50.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:20:50.455 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:20:50.508 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T14:20:50.508 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T14:20:50.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:20:50.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T14:20:50.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T14:20:50.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:20:50.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T14:20:50.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T14:20:50.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:20:50.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T14:20:50.980 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:50.982 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:20:51.035 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T14:20:51.035 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T14:20:51.035 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T14:20:51.038 INFO:tasks.ceph.mon.a:Started 2024-09-21T14:20:51.038 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T14:20:51.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T14:20:51.040 INFO:tasks.ceph.mon.c:Started 2024-09-21T14:20:51.040 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T14:20:51.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T14:20:51.077 INFO:tasks.ceph.mon.b:Started 2024-09-21T14:20:51.077 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T14:20:51.077 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T14:20:51.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T14:20:51.080 INFO:tasks.ceph.mgr.y:Started 2024-09-21T14:20:51.080 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T14:20:51.080 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T14:20:51.082 INFO:tasks.ceph.mgr.x:Started 2024-09-21T14:20:51.082 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T14:20:51.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T14:20:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-09-21T14:20:56.447 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T14:20:56.447 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T14:20:56.447 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:56.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T14:20:56.465 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.465+0000 7f35c0634040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T14:20:56.472 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:56.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T14:20:56.481 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.481+0000 7fa952e2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T14:20:56.505 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:56.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T14:20:56.507 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.507+0000 7f35c0634040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T14:20:56.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.524+0000 7fa952e2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T14:20:56.549 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-21T14:20:56.549 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T14:20:56.615 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:56.616 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T14:20:56.644 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:56.644 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T14:20:56.653 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.653+0000 7f35c0634040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T14:20:56.674 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.674+0000 7fa952e2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T14:20:56.677 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:56.677 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T14:20:56.706 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.706+0000 7f35c0634040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T14:20:56.724 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.724+0000 7fa952e2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T14:20:56.730 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T14:20:56.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T14:20:56.795 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 4cd103e2-3667-49ee-9ab4-f3b9e9a18d18 0 2024-09-21T14:20:56.877 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.877+0000 7f35c0634040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:20:56.885 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.885+0000 7fa952e2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:20:56.993 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:56.993+0000 7f35c0634040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T14:20:56.994 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:56.994+0000 7fa952e2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T14:20:57.016 INFO:teuthology.orchestra.run.smithi161.stdout:0 2024-09-21T14:20:57.027 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new b7cc4ae1-1649-4592-91e7-4814ad9aa260 1 2024-09-21T14:20:57.331 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:57.331+0000 7f35c0634040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T14:20:57.334 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:57.334+0000 7fa952e2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T14:20:57.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T14:20:57.370+0000 7f73b7fff640 0 --2- 172.21.15.161:0/1565983481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c00ac640 0x7f73c00a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T14:20:57.389 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:57.389+0000 7fa952e2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T14:20:57.398 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:57.398+0000 7f35c0634040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T14:20:57.506 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-09-21T14:20:57.517 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new adda4736-6878-49a4-ae7c-77b97c866703 2 2024-09-21T14:20:57.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:57.536+0000 7fa952e2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T14:20:57.591 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:57.591+0000 7f35c0634040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T14:20:57.714 INFO:teuthology.orchestra.run.smithi161.stdout:2 2024-09-21T14:20:57.725 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new af765f77-37b5-4efd-89f0-4dfe5ef0978f 3 2024-09-21T14:20:57.899 INFO:teuthology.orchestra.run.smithi161.stdout:3 2024-09-21T14:20:57.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c 4 2024-09-21T14:20:58.091 INFO:teuthology.orchestra.run.smithi161.stdout:4 2024-09-21T14:20:58.102 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 5daf6d5d-18e7-475d-87ae-4b15965a8192 5 2024-09-21T14:20:58.284 INFO:teuthology.orchestra.run.smithi161.stdout:5 2024-09-21T14:20:58.296 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new e75e58a4-a37f-4029-b4fb-7ba22db2d6f0 6 2024-09-21T14:20:58.476 INFO:teuthology.orchestra.run.smithi161.stdout:6 2024-09-21T14:20:58.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new c0378ee4-c4f7-4430-ac1e-5740e2e8bfae 7 2024-09-21T14:20:58.489 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:58.489+0000 7fa952e2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T14:20:58.540 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:58.539+0000 7fa952e2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T14:20:58.589 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:58.589+0000 7f35c0634040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T14:20:58.643 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:58.643+0000 7f35c0634040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T14:20:58.667 INFO:teuthology.orchestra.run.smithi161.stdout:7 2024-09-21T14:20:58.678 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T14:20:58.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T14:20:58.680 INFO:tasks.ceph.osd.0:Started 2024-09-21T14:20:58.681 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T14:20:58.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T14:20:58.683 INFO:tasks.ceph.osd.1:Started 2024-09-21T14:20:58.683 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T14:20:58.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T14:20:58.685 INFO:tasks.ceph.osd.2:Started 2024-09-21T14:20:58.685 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T14:20:58.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T14:20:58.690 INFO:tasks.ceph.osd.3:Started 2024-09-21T14:20:58.690 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T14:20:58.691 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T14:20:58.693 INFO:tasks.ceph.osd.4:Started 2024-09-21T14:20:58.693 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T14:20:58.693 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T14:20:58.695 INFO:tasks.ceph.osd.5:Started 2024-09-21T14:20:58.695 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T14:20:58.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T14:20:58.697 INFO:tasks.ceph.osd.6:Started 2024-09-21T14:20:58.697 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T14:20:58.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T14:20:58.699 INFO:tasks.ceph.osd.7:Started 2024-09-21T14:20:58.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:20:58.895 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:20:58.895 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":10,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:20:58.662769+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T14:20:58.662769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:20:58.905 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T14:20:58.905 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T14:20:58.948 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:58.947+0000 7fa952e2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-21T14:20:59.031 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.031+0000 7fa952e2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T14:20:59.049 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.049+0000 7f35c0634040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-21T14:20:59.133 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.133+0000 7f35c0634040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T14:20:59.136 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.136+0000 7fa952e2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:20:59.181 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.181+0000 7fa952e2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T14:20:59.238 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.237+0000 7f35c0634040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:20:59.283 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.283+0000 7f35c0634040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T14:20:59.287 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:20:59.287+0000 7ff22f60d640 -1 Falling back to public interface 2024-09-21T14:20:59.289 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:20:59.289+0000 7f8aad60b640 -1 Falling back to public interface 2024-09-21T14:20:59.291 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.291+0000 7fa952e2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-21T14:20:59.295 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:20:59.295+0000 7f57c6487640 -1 Falling back to public interface 2024-09-21T14:20:59.297 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:20:59.297+0000 7f46320ef640 -1 Falling back to public interface 2024-09-21T14:20:59.298 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:20:59.298+0000 7fc344212640 -1 Falling back to public interface 2024-09-21T14:20:59.306 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:20:59.306+0000 7f7fccc2c640 -1 Falling back to public interface 2024-09-21T14:20:59.315 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:20:59.315+0000 7ff33d32b640 -1 Falling back to public interface 2024-09-21T14:20:59.324 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:20:59.324+0000 7f09e77a8640 -1 Falling back to public interface 2024-09-21T14:20:59.346 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.345+0000 7fa952e2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T14:20:59.391 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.391+0000 7fa952e2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T14:20:59.397 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.397+0000 7f35c0634040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-21T14:20:59.450 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.450+0000 7f35c0634040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T14:20:59.493 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.493+0000 7f35c0634040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T14:20:59.500 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.500+0000 7fa952e2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T14:20:59.550 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.550+0000 7fa952e2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T14:20:59.603 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.602+0000 7fa952e2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T14:20:59.605 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.605+0000 7f35c0634040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T14:20:59.654 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.654+0000 7f35c0634040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T14:20:59.670 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.670+0000 7fa952e2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T14:20:59.704 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.704+0000 7f35c0634040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T14:20:59.711 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T14:20:59.714 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.714+0000 7fa952e2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T14:20:59.760 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:20:59.759+0000 7fa952e2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T14:20:59.771 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.771+0000 7f35c0634040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T14:20:59.815 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.815+0000 7f35c0634040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T14:20:59.860 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:20:59.860+0000 7f35c0634040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T14:20:59.869 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-21T14:20:59.869 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":10,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:20:58.662769+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T14:20:58.662769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:20:59.879 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T14:21:00.407 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:21:00.407+0000 7fa952e2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T14:21:00.505 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-09-21T14:21:00.505+0000 7f35c0634040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T14:21:01.422 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:21:01.422+0000 7fa8f3019640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T14:21:01.422 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-09-21T14:21:01.422+0000 7fa8f3019640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T14:21:02.744 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:02.744+0000 7f8aad60b640 -1 osd.4 0 log_to_monitors true 2024-09-21T14:21:02.749 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:02.749+0000 7ff22f60d640 -1 osd.0 0 log_to_monitors true 2024-09-21T14:21:02.781 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:02.782+0000 7fc344212640 -1 osd.6 0 log_to_monitors true 2024-09-21T14:21:02.801 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:02.801+0000 7f7fccc2c640 -1 osd.2 0 log_to_monitors true 2024-09-21T14:21:02.806 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:02.807+0000 7f46320ef640 -1 osd.1 0 log_to_monitors true 2024-09-21T14:21:02.807 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:02.807+0000 7f57c6487640 -1 osd.5 0 log_to_monitors true 2024-09-21T14:21:02.808 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:02.808+0000 7ff33d32b640 -1 osd.3 0 log_to_monitors true 2024-09-21T14:21:02.827 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:02.827+0000 7f09e77a8640 -1 osd.7 0 log_to_monitors true 2024-09-21T14:21:05.516 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:05.515+0000 7ff22b01a640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T14:21:05.534 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:05.534+0000 7ff222e3b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:05.545+0000 7f09e3735640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T14:21:05.554 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:05.554+0000 7f462e07c640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T14:21:05.554 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:05.554+0000 7f7fc8238640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T14:21:05.561 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:05.561+0000 7f09db556640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.563 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:05.563+0000 7ff3392b8640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T14:21:05.566 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:05.566+0000 7fc33fc21640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T14:21:05.566 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:05.566+0000 7f4625e9d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.567 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:05.567+0000 7f7fc0059640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.571 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:05.571+0000 7f57c2414640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T14:21:05.576 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:05.576+0000 7ff3310d9640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.582 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:05.582+0000 7f57ba235640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.582 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:05.582+0000 7fc337a42640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:05.585 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:05.585+0000 7f8aa902f640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T14:21:05.596 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:05.596+0000 7f8aa0e50640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:21:06.685 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T14:21:06.958 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-21T14:21:06.959 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":13,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:21:06.519050+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:20:58.662769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:21:06.969 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T14:21:06.969 INFO:tasks.ceph:Creating RBD pool 2024-09-21T14:21:06.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T14:21:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'rbd' created 2024-09-21T14:21:07.537 DEBUG:teuthology.orchestra.run.smithi073:> rbd --cluster ceph pool init rbd 2024-09-21T14:21:10.677 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T14:21:10.677 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T14:21:10.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T14:21:10.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T14:21:10.678 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T14:21:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:11.011 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":4,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2610434360},{"type":"v1","addr":"172.21.15.73:6833","nonce":2610434360}]},"active_addr":"172.21.15.73:6833/2610434360","active_change":"2024-09-21T14:21:00.410229+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3582297441}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2394201077}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1709219647}]}]} 2024-09-21T14:21:11.015 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T14:21:11.015 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T14:21:11.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:21:11.284 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:11.285 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:21:10.595310+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:20:58.662769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-21T14:21:07.255609+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-21T14:21:08.449794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.729613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.843490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.767547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.797830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.760985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.830324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.747245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.849223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:21:11.297 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T14:21:11.297 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:21:11.571 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:11.571 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:21:10.595310+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:20:58.662769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-21T14:21:07.255609+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-21T14:21:08.449794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.729613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.843490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.767547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.797830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.760985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.830324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.747245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.849223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:21:11.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T14:21:11.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T14:21:11.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T14:21:11.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T14:21:11.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T14:21:11.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T14:21:11.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T14:21:11.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T14:21:11.812 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:11.812 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:21:11.823 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:11.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:21:11.826 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:11.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:21:11.827 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:11.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:21:11.828 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:11.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:21:11.830 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:11.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:21:11.831 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:11.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:21:11.832 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:11.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:21:12.321 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:12.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-09-21T14:21:12.360 INFO:teuthology.orchestra.run.smithi073.stdout:55834574849 2024-09-21T14:21:12.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.6 2024-09-21T14:21:12.393 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:12.401 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:12.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-09-21T14:21:12.409 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:12.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-09-21T14:21:12.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-09-21T14:21:12.452 INFO:teuthology.orchestra.run.smithi073.stdout:55834574849 2024-09-21T14:21:12.453 INFO:teuthology.orchestra.run.smithi073.stdout:55834574849 2024-09-21T14:21:12.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.5 2024-09-21T14:21:12.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2024-09-21T14:21:12.486 INFO:teuthology.orchestra.run.smithi073.stdout:55834574849 2024-09-21T14:21:12.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2024-09-21T14:21:13.340 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:21:13.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:21:13.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:21:13.415 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:21:13.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:21:13.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:21:13.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:21:13.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:21:13.847 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:13.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2024-09-21T14:21:13.872 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:13.903 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:13.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.6 2024-09-21T14:21:13.922 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:13.948 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:13.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.5 2024-09-21T14:21:13.961 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:13.977 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:13.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-09-21T14:21:13.993 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:13.995 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:14.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.2 2024-09-21T14:21:14.008 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:14.010 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:14.012 INFO:teuthology.orchestra.run.smithi073.stdout:55834574850 2024-09-21T14:21:14.014 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2024-09-21T14:21:14.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-09-21T14:21:14.025 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:14.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.4 2024-09-21T14:21:14.027 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:14.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-09-21T14:21:14.029 DEBUG:teuthology.parallel:result is None 2024-09-21T14:21:14.029 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T14:21:14.029 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:14.337 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:14.337 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:14.349 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-21T14:21:12.419292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217688,"kb_used_data":1404,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757032,"statfs":{"total":773094113280,"available":772871200768,"internally_reserved":0,"allocated":1437696,"data_stored":1586684,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671401+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:10.671401+0000","last_peered":"2024-09-21T14:21:10.671401+0000","last_clean":"2024-09-21T14:21:10.671401+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:10.671401+0000","last_undegraded":"2024-09-21T14:21:10.671401+0000","last_fullsized":"2024-09-21T14:21:10.671401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745650+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:10.745650+0000","last_peered":"2024-09-21T14:21:10.745650+0000","last_clean":"2024-09-21T14:21:10.745650+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:10.745650+0000","last_undegraded":"2024-09-21T14:21:10.745650+0000","last_fullsized":"2024-09-21T14:21:10.745650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011596+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:11.011596+0000","last_peered":"2024-09-21T14:21:11.011596+0000","last_clean":"2024-09-21T14:21:11.011596+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:11.011596+0000","last_undegraded":"2024-09-21T14:21:11.011596+0000","last_fullsized":"2024-09-21T14:21:11.011596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:11.011737+0000","last_peered":"2024-09-21T14:21:11.011737+0000","last_clean":"2024-09-21T14:21:11.011737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:11.011737+0000","last_undegraded":"2024-09-21T14:21:11.011737+0000","last_fullsized":"2024-09-21T14:21:11.011737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.757512+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:10.757512+0000","last_peered":"2024-09-21T14:21:10.757512+0000","last_clean":"2024-09-21T14:21:10.757512+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:10.757512+0000","last_undegraded":"2024-09-21T14:21:10.757512+0000","last_fullsized":"2024-09-21T14:21:10.757512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.669229+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:10.669229+0000","last_peered":"2024-09-21T14:21:10.669229+0000","last_clean":"2024-09-21T14:21:10.669229+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:10.669229+0000","last_undegraded":"2024-09-21T14:21:10.669229+0000","last_fullsized":"2024-09-21T14:21:10.669229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.674039+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:10.674039+0000","last_peered":"2024-09-21T14:21:10.674039+0000","last_clean":"2024-09-21T14:21:10.674039+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:10.674039+0000","last_undegraded":"2024-09-21T14:21:10.674039+0000","last_fullsized":"2024-09-21T14:21:10.674039+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745649+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:10.745649+0000","last_peered":"2024-09-21T14:21:10.745649+0000","last_clean":"2024-09-21T14:21:10.745649+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:10.745649+0000","last_undegraded":"2024-09-21T14:21:10.745649+0000","last_fullsized":"2024-09-21T14:21:10.745649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671402+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:10.671402+0000","last_peered":"2024-09-21T14:21:10.671402+0000","last_clean":"2024-09-21T14:21:10.671402+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:10.671402+0000","last_undegraded":"2024-09-21T14:21:10.671402+0000","last_fullsized":"2024-09-21T14:21:10.671402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":229376,"data_stored":642693,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":233472,"data_stored":643679,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:14.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:14.622 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:14.633 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-21T14:21:14.419612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217712,"kb_used_data":1440,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757008,"statfs":{"total":773094113280,"available":772871176192,"internally_reserved":0,"allocated":1474560,"data_stored":1605290,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671401+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:10.671401+0000","last_peered":"2024-09-21T14:21:10.671401+0000","last_clean":"2024-09-21T14:21:10.671401+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:10.671401+0000","last_undegraded":"2024-09-21T14:21:10.671401+0000","last_fullsized":"2024-09-21T14:21:10.671401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745650+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:10.745650+0000","last_peered":"2024-09-21T14:21:10.745650+0000","last_clean":"2024-09-21T14:21:10.745650+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:10.745650+0000","last_undegraded":"2024-09-21T14:21:10.745650+0000","last_fullsized":"2024-09-21T14:21:10.745650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011596+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:11.011596+0000","last_peered":"2024-09-21T14:21:11.011596+0000","last_clean":"2024-09-21T14:21:11.011596+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:11.011596+0000","last_undegraded":"2024-09-21T14:21:11.011596+0000","last_fullsized":"2024-09-21T14:21:11.011596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:11.011737+0000","last_peered":"2024-09-21T14:21:11.011737+0000","last_clean":"2024-09-21T14:21:11.011737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:11.011737+0000","last_undegraded":"2024-09-21T14:21:11.011737+0000","last_fullsized":"2024-09-21T14:21:11.011737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.757512+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:10.757512+0000","last_peered":"2024-09-21T14:21:10.757512+0000","last_clean":"2024-09-21T14:21:10.757512+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:10.757512+0000","last_undegraded":"2024-09-21T14:21:10.757512+0000","last_fullsized":"2024-09-21T14:21:10.757512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.669229+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:10.669229+0000","last_peered":"2024-09-21T14:21:10.669229+0000","last_clean":"2024-09-21T14:21:10.669229+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:10.669229+0000","last_undegraded":"2024-09-21T14:21:10.669229+0000","last_fullsized":"2024-09-21T14:21:10.669229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.674039+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:10.674039+0000","last_peered":"2024-09-21T14:21:10.674039+0000","last_clean":"2024-09-21T14:21:10.674039+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:10.674039+0000","last_undegraded":"2024-09-21T14:21:10.674039+0000","last_fullsized":"2024-09-21T14:21:10.674039+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745649+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:10.745649+0000","last_peered":"2024-09-21T14:21:10.745649+0000","last_clean":"2024-09-21T14:21:10.745649+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:10.745649+0000","last_undegraded":"2024-09-21T14:21:10.745649+0000","last_fullsized":"2024-09-21T14:21:10.745649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671402+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:10.671402+0000","last_peered":"2024-09-21T14:21:10.671402+0000","last_clean":"2024-09-21T14:21:10.671402+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:10.671402+0000","last_undegraded":"2024-09-21T14:21:10.671402+0000","last_fullsized":"2024-09-21T14:21:10.671402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":229376,"data_stored":642693,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":233472,"data_stored":643679,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:14.634 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T14:21:14.635 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T14:21:14.635 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T14:21:14.635 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T14:21:14.937 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:14.938 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T14:21:14.950 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T14:21:14.950 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T14:21:14.961 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T14:21:14.962 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}} 2024-09-21T14:21:14.962 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T14:21:14.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T14:21:15.251 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T14:21:15.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T14:21:15.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:15.523 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:15.523 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:15.534 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-21T14:21:14.419612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217712,"kb_used_data":1440,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757008,"statfs":{"total":773094113280,"available":772871176192,"internally_reserved":0,"allocated":1474560,"data_stored":1605290,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671401+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:10.671401+0000","last_peered":"2024-09-21T14:21:10.671401+0000","last_clean":"2024-09-21T14:21:10.671401+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:10.671401+0000","last_undegraded":"2024-09-21T14:21:10.671401+0000","last_fullsized":"2024-09-21T14:21:10.671401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745650+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:10.745650+0000","last_peered":"2024-09-21T14:21:10.745650+0000","last_clean":"2024-09-21T14:21:10.745650+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:10.745650+0000","last_undegraded":"2024-09-21T14:21:10.745650+0000","last_fullsized":"2024-09-21T14:21:10.745650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011596+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:11.011596+0000","last_peered":"2024-09-21T14:21:11.011596+0000","last_clean":"2024-09-21T14:21:11.011596+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:11.011596+0000","last_undegraded":"2024-09-21T14:21:11.011596+0000","last_fullsized":"2024-09-21T14:21:11.011596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:11.011737+0000","last_peered":"2024-09-21T14:21:11.011737+0000","last_clean":"2024-09-21T14:21:11.011737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:11.011737+0000","last_undegraded":"2024-09-21T14:21:11.011737+0000","last_fullsized":"2024-09-21T14:21:11.011737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.757512+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:10.757512+0000","last_peered":"2024-09-21T14:21:10.757512+0000","last_clean":"2024-09-21T14:21:10.757512+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:10.757512+0000","last_undegraded":"2024-09-21T14:21:10.757512+0000","last_fullsized":"2024-09-21T14:21:10.757512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.669229+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:10.669229+0000","last_peered":"2024-09-21T14:21:10.669229+0000","last_clean":"2024-09-21T14:21:10.669229+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:10.669229+0000","last_undegraded":"2024-09-21T14:21:10.669229+0000","last_fullsized":"2024-09-21T14:21:10.669229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.674039+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:10.674039+0000","last_peered":"2024-09-21T14:21:10.674039+0000","last_clean":"2024-09-21T14:21:10.674039+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:10.674039+0000","last_undegraded":"2024-09-21T14:21:10.674039+0000","last_fullsized":"2024-09-21T14:21:10.674039+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745649+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:10.745649+0000","last_peered":"2024-09-21T14:21:10.745649+0000","last_clean":"2024-09-21T14:21:10.745649+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:10.745649+0000","last_undegraded":"2024-09-21T14:21:10.745649+0000","last_fullsized":"2024-09-21T14:21:10.745649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671402+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:10.671402+0000","last_peered":"2024-09-21T14:21:10.671402+0000","last_clean":"2024-09-21T14:21:10.671402+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:10.671402+0000","last_undegraded":"2024-09-21T14:21:10.671402+0000","last_fullsized":"2024-09-21T14:21:10.671402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":229376,"data_stored":642693,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":233472,"data_stored":643679,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:15.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:15.805 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-21T14:21:14.419612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217712,"kb_used_data":1440,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757008,"statfs":{"total":773094113280,"available":772871176192,"internally_reserved":0,"allocated":1474560,"data_stored":1605290,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671401+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:10.671401+0000","last_peered":"2024-09-21T14:21:10.671401+0000","last_clean":"2024-09-21T14:21:10.671401+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:10.671401+0000","last_undegraded":"2024-09-21T14:21:10.671401+0000","last_fullsized":"2024-09-21T14:21:10.671401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745650+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:10.745650+0000","last_peered":"2024-09-21T14:21:10.745650+0000","last_clean":"2024-09-21T14:21:10.745650+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:10.745650+0000","last_undegraded":"2024-09-21T14:21:10.745650+0000","last_fullsized":"2024-09-21T14:21:10.745650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011596+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:11.011596+0000","last_peered":"2024-09-21T14:21:11.011596+0000","last_clean":"2024-09-21T14:21:11.011596+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:11.011596+0000","last_undegraded":"2024-09-21T14:21:11.011596+0000","last_fullsized":"2024-09-21T14:21:11.011596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:11.011737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:11.011737+0000","last_peered":"2024-09-21T14:21:11.011737+0000","last_clean":"2024-09-21T14:21:11.011737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:11.011737+0000","last_undegraded":"2024-09-21T14:21:11.011737+0000","last_fullsized":"2024-09-21T14:21:11.011737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.757512+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:10.757512+0000","last_peered":"2024-09-21T14:21:10.757512+0000","last_clean":"2024-09-21T14:21:10.757512+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:10.757512+0000","last_undegraded":"2024-09-21T14:21:10.757512+0000","last_fullsized":"2024-09-21T14:21:10.757512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.669229+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:10.669229+0000","last_peered":"2024-09-21T14:21:10.669229+0000","last_clean":"2024-09-21T14:21:10.669229+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:10.669229+0000","last_undegraded":"2024-09-21T14:21:10.669229+0000","last_fullsized":"2024-09-21T14:21:10.669229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.674039+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:10.674039+0000","last_peered":"2024-09-21T14:21:10.674039+0000","last_clean":"2024-09-21T14:21:10.674039+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:10.674039+0000","last_undegraded":"2024-09-21T14:21:10.674039+0000","last_fullsized":"2024-09-21T14:21:10.674039+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.745649+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:10.745649+0000","last_peered":"2024-09-21T14:21:10.745649+0000","last_clean":"2024-09-21T14:21:10.745649+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:10.745649+0000","last_undegraded":"2024-09-21T14:21:10.745649+0000","last_fullsized":"2024-09-21T14:21:10.745649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-21T14:21:10.671402+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:10.671402+0000","last_peered":"2024-09-21T14:21:10.671402+0000","last_clean":"2024-09-21T14:21:10.671402+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:10.671402+0000","last_undegraded":"2024-09-21T14:21:10.671402+0000","last_fullsized":"2024-09-21T14:21:10.671402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":229376,"data_stored":642693,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":233472,"data_stored":643679,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:15.818 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T14:21:15.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 17 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:fsid da87574b-7df0-407c-8805-347c49d2d055 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-21T14:20:56.341644+0000 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-09-21T14:21:10.595310+0000 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:crush_version 4 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:full_ratio 0.95 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:backfillfull_ratio 0.9 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:nearfull_ratio 0.85 2024-09-21T14:21:16.086 INFO:teuthology.orchestra.run.smithi073.stdout:require_min_compat_client luminous 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:min_compat_client jewel 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:require_osd_release reef 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:stretch_mode_enabled false 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: removed_snaps_queue [2~1] 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:max_osd 8 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/3182191515,v1:172.21.15.73:6801/3182191515] [v2:172.21.15.73:6802/3182191515,v1:172.21.15.73:6803/3182191515] exists,up 4cd103e2-3667-49ee-9ab4-f3b9e9a18d18 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/3526445386,v1:172.21.15.73:6809/3526445386] [v2:172.21.15.73:6810/3526445386,v1:172.21.15.73:6811/3526445386] exists,up b7cc4ae1-1649-4592-91e7-4814ad9aa260 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/3383683073,v1:172.21.15.73:6817/3383683073] [v2:172.21.15.73:6818/3383683073,v1:172.21.15.73:6819/3383683073] exists,up adda4736-6878-49a4-ae7c-77b97c866703 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/3204248728,v1:172.21.15.73:6825/3204248728] [v2:172.21.15.73:6826/3204248728,v1:172.21.15.73:6827/3204248728] exists,up af765f77-37b5-4efd-89f0-4dfe5ef0978f 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/3444590304,v1:172.21.15.161:6801/3444590304] [v2:172.21.15.161:6802/3444590304,v1:172.21.15.161:6803/3444590304] exists,up 8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/4067730938,v1:172.21.15.161:6809/4067730938] [v2:172.21.15.161:6810/4067730938,v1:172.21.15.161:6811/4067730938] exists,up 5daf6d5d-18e7-475d-87ae-4b15965a8192 2024-09-21T14:21:16.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2060683966,v1:172.21.15.161:6817/2060683966] [v2:172.21.15.161:6818/2060683966,v1:172.21.15.161:6819/2060683966] exists,up e75e58a4-a37f-4029-b4fb-7ba22db2d6f0 2024-09-21T14:21:16.088 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2285860342,v1:172.21.15.161:6825/2285860342] [v2:172.21.15.161:6826/2285860342,v1:172.21.15.161:6827/2285860342] exists,up c0378ee4-c4f7-4430-ac1e-5740e2e8bfae 2024-09-21T14:21:16.098 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/3182191515,v1:172.21.15.73:6801/3182191515] [v2:172.21.15.73:6802/3182191515,v1:172.21.15.73:6803/3182191515] exists,up 4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/3526445386,v1:172.21.15.73:6809/3526445386] [v2:172.21.15.73:6810/3526445386,v1:172.21.15.73:6811/3526445386] exists,up b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'osd.2 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/3383683073,v1:172.21.15.73:6817/3383683073] [v2:172.21.15.73:6818/3383683073,v1:172.21.15.73:6819/3383683073] exists,up adda4736-6878-49a4-ae7c-77b97c866703', 'osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/3204248728,v1:172.21.15.73:6825/3204248728] [v2:172.21.15.73:6826/3204248728,v1:172.21.15.73:6827/3204248728] exists,up af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/3444590304,v1:172.21.15.161:6801/3444590304] [v2:172.21.15.161:6802/3444590304,v1:172.21.15.161:6803/3444590304] exists,up 8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'osd.5 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/4067730938,v1:172.21.15.161:6809/4067730938] [v2:172.21.15.161:6810/4067730938,v1:172.21.15.161:6811/4067730938] exists,up 5daf6d5d-18e7-475d-87ae-4b15965a8192', 'osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2060683966,v1:172.21.15.161:6817/2060683966] [v2:172.21.15.161:6818/2060683966,v1:172.21.15.161:6819/2060683966] exists,up e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'osd.7 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2285860342,v1:172.21.15.161:6825/2285860342] [v2:172.21.15.161:6826/2285860342,v1:172.21.15.161:6827/2285860342] exists,up c0378ee4-c4f7-4430-ac1e-5740e2e8bfae'] 2024-09-21T14:21:16.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mon.a", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster": "ceph", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket_mode": "", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_client_required": "cephx, none", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_cluster_required": "cephx", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_debug": "true", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T14:21:16.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_required": "cephx", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_supported": "cephx", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio": "true", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_reap_max": "16", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_async_discard": "false", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_block_size": "4096", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio": "false", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_enable_discard": "false", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry": "3", 2024-09-21T14:21:16.199 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash": "0", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_hipri": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_type": "", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_allocator": "hybrid", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_buffered_io": "true", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T14:21:16.200 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery": "false", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_sync_write": "false", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T14:21:16.201 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocator": "hybrid", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_create": "true", 2024-09-21T14:21:16.202 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_create": "false", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_path": "", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_size": "0", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_path": "", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_size": "107374182400", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_create": "false", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_path": "", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs": "true", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T14:21:16.203 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune": "true", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size": "0", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_type": "2q", 2024-09-21T14:21:16.204 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_clone_cow": "true", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_mode": "none", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_freelist": "false", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T14:21:16.205 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T14:21:16.206 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fail_eio": "false", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T14:21:16.207 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T14:21:16.208 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size": "0", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T14:21:16.209 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T14:21:16.210 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_mem": "512", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_tracing": "false", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T14:21:16.211 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T14:21:16.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_version": "2", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_signatures": "false", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_version": "2", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_signatures": "false", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_version": "2", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_sign_messages": "true", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "chdir": "", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_acl_type": "", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_asio_thread_count": "2", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_mid": "0.750000", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_size": "16384", 2024-09-21T14:21:16.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_caps_release_delay": "5", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_check_pool_perm": "true", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_force_sync_read": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_getattr_caps": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_inject_features": "", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_remount": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_dirsize_rbytes": "true", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_force_lazyio": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_fs": "", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_release_failure": "false", 2024-09-21T14:21:16.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_inline_size": "4096", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mds_namespace": "", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_metadata": "", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_gid": "-1", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_timeout": "300", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_uid": "-1", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mountpoint": "/", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_notify_timeout": "10", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc": "true", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_objects": "1000", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_size": "209715200", 2024-09-21T14:21:16.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_permissions": "true", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota": "true", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota_df": "true", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_bytes": "0", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_periods": "4", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_min": "131072", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_reconnect_stale": "false", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_shutdown_timeout": "30", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_snapdir": ".snap", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_tick_interval": "1", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_trace": "", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_faked_inos": "false", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_random_mds": "false", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog": "false", 2024-09-21T14:21:16.216 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_port": "12201", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_monitors": "default=true", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog": "false", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_level": "info", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "-", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network": "", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network_interface": "", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_isal": "false", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_level": "5", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zstd_level": "1", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T14:21:16.217 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_seastar_smp": "1", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location": "", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook": "", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook_timeout": "10", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "daemonize": "false", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "daos_pool": "tank", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_alienstore": "0/5", 2024-09-21T14:21:16.218 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok": "1/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok_assert_abort": "false", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_auth": "20/20", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bdev": "1/3", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluefs": "1/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluestore": "1/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_buffer": "0/1", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephsqlite": "0/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_client": "0/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_compressor": "1/5", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_context": "0/1", 2024-09-21T14:21:16.219 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crush": "1/1", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crypto": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cyanstore": "0/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_dpdk": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_eventtrace": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filer": "0/1", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filestore": "1/3", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_finisher": "1/1", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_fuse": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_javaclient": "1/5", 2024-09-21T14:21:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journal": "1/3", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journaler": "0/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_kstore": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_leveldb": "4/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_lockdep": "0/1", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mclock": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_balancer": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_locker": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_migrator": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_memstore": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgr": "2/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgrc": "1/5", 2024-09-21T14:21:16.221 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mon": "20/20", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_monc": "0/10", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ms": "1/1", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_none": "0/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objclass": "0/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objectcacher": "0/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objecter": "0/1", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_optracker": "0/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_osd": "1/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_paxos": "20/20", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_perfcounter": "1/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_prioritycache": "1/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rados": "0/5", 2024-09-21T14:21:16.222 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_replay": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_refs": "0/0", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_reserver": "1/1", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_access": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_flight": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_sync": "1/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rocksdb": "4/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_backref": "0/5", 2024-09-21T14:21:16.223 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cache": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_device": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_epm": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_journal": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_lba": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_odata": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_omap": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_onode": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_t": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_tm": "0/5", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_striper": "0/1", 2024-09-21T14:21:16.224 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_test": "0/5", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_throttle": "1/1", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_timer": "0/1", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_tp": "0/5", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "device_failure_prediction_mode": "none", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_graylog": "false", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_journald": "false", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_stderr": "true", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_syslog": "false", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event_tracing": "false", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T14:21:16.225 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_http_port": "9926", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_prio_limit": "5", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sort_metrics": "true", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_stats_period": "5", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "fake_statfs_for_testing": "0", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "fatal_signal_handlers": "true", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_purge_ops": "10", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_truncate_ops": "128", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_blackhole": "false", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_snap": "true", 2024-09-21T14:21:16.226 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_caller_concurrency": "10", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_omap_check": "false", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_verify_split": "false", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_dump_file": "", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fadvise": "true", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fail_eio": "true", 2024-09-21T14:21:16.227 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_size": "128", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap": "false", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_inject_stall": "0", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_parallel": "false", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_trailing": "false", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_writeahead": "false", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_kill_at": "0", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T14:21:16.228 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_merge_threshold": "-10", 2024-09-21T14:21:16.229 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_odsync_write": "false", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend_path": "", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_threads": "2", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_punch_hole": "false", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T14:21:16.230 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_ops": "50", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_seek_data_hole": "false", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc": "false", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_splice": "false", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_multiple": "2", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_rand_factor": "20", 2024-09-21T14:21:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_update_to": "1000", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T14:21:16.232 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_xfs_extsize": "false", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_zfs_snap": "false", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fio_dir": "/tmp/fio", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_allow_other": "true", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_big_writes": "true", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_debug": "false", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_default_permissions": "false", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_disable_pagecache": "false", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_max_write": "0", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_multithreaded": "true", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_require_active_mds": "true", 2024-09-21T14:21:16.233 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_set_user_groups": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_move": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_read": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_write": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_target_name": "ceph", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_file": "", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_inject_failure": "0", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_interval": "5", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "host": "smithi073", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T14:21:16.234 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "inject_early_sigterm": "false", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_agent_port": "6799", 2024-09-21T14:21:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_tracing_enable": "false", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_aio": "true", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_align_min_size": "65536", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_align": "true", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_size": "4096", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_dio": "true", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_discard": "false", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_force_aio": "false", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_ignore_corruption": "false", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_entries": "100", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_replay_from": "0", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T14:21:16.236 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_write_header_frequency": "0", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_zero_on_create": "false", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prefetch_periods": "10", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prezero_periods": "5", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_write_head_interval": "15", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "keyfile": "", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_backend": "rocksdb", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_bytes": "67108864", 2024-09-21T14:21:16.237 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_ops": "512", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_onode_map_size": "1024", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_transaction": "false", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_block_size": "65536", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_bloom_size": "0", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_cache_size": "536870912", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_compact_on_mount": "false", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_compression": "false", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_log": "", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_max_open_files": "0", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_paranoid": "false", 2024-09-21T14:21:16.238 INFO:teuthology.orchestra.run.smithi073.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "librados_thread_count": "2", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep": "false", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep_force_backtrace": "false", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_coarse_timestamps": "true", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_flush_on_exit": "false", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_port": "12201", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_new": "1000", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_recent": "10000", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stderr_prefix": "", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": "true", 2024-09-21T14:21:16.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_graylog": "false", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_journald": "false", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_stderr": "false", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_syslog": "false", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_action_on_write_error": "1", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_alternate_name_max": "8192", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_asio_thread_count": "2", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_export_pin": "true", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T14:21:16.240 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_interval": "10", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max": "-1", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max_until": "-1", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_merge_size": "50", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_mode": "0", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_bits": "3", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T14:21:16.241 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_size": "10000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_mid": "0.700000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_interval": "1", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T14:21:16.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_auth_pins": "false", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_frag": "false", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_scatterstat": "false", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_subtrees": "false", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_default_dir_hash": "2", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_defer_session_stale": "true", 2024-09-21T14:21:16.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_entries": "0", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_prefetch": "true", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_early_reply": "true", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enable_op_tracker": "true", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enforce_unique_name": "true", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T14:21:16.244 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_health_dummy": "false", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T14:21:16.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_join_fs": "", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_journal_format": "1", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_create_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_export_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_import_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_link_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_openc_at": "0", 2024-09-21T14:21:16.246 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_rename_at": "0", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_events": "-1", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_segments": "128", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_pause": "false", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_segment_size": "0", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_requests": "100000", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_export_size": "20971520", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_file_recover": "32", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_files": "64", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops": "8192", 2024-09-21T14:21:16.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_metrics_update_interval": "2", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_per_client": "100", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_numa_node": "-1", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_duration": "600", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_size": "20", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_log_threshold": "5", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_grace": "15", 2024-09-21T14:21:16.248 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_interval": "5", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_caps": "30000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_interval": "1.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_gid": "0", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_uid": "0", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T14:21:16.249 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_shutdown_check": "0", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_skip_ino": "0", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_min_uid": "0", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_rstat": "false", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T14:21:16.250 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_symlink_recovery": "true", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_exports": "0", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_fragments": "0", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_tick_interval": "5.000000", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_valgrind_exit": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_backtrace": "1", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_scatter": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_sessions": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool_debug": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_set": "false", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_size": "65536", 2024-09-21T14:21:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_bytes": "134217728", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_messages": "512", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_disabled_modules": "", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_messages": "128", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_messages": "128", 2024-09-21T14:21:16.252 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_messages": "8192", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_pool": "true", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_standby_modules": "true", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_period": "5", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_threshold": "5", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_tick_period": "2", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_delete": "true", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T14:21:16.253 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_bytes": "104857600", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T14:21:16.254 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T14:21:16.255 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_start": "false", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_trim": "true", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cpu_threads": "4", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_crit": "5", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_warn": "5", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T14:21:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_json": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_extra_checks": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_quincy": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T14:21:16.257 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_default_strategy": "1", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_timeout": "5.000000", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_enable_op_tracker": "true", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_fake_pool_delete": "false", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_force_quorum_join": "false", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_log_update_period": "5", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_max_detail": "50", 2024-09-21T14:21:16.258 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog": "true", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host": "172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],172.21.15.161", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host_override": "", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_initial_members": "", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease": "5.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T14:21:16.259 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_full_interval": "50", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max": "10000", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max_summary": "50", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_epochs": "500", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_osd": "10000", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T14:21:16.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_autotune": "true", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_target": "2147483648", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_digest_period": "5", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_complaint_time": "30", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_duration": "600", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_size": "20", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_log_threshold": "5", 2024-09-21T14:21:16.261 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size": "500", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T14:21:16.262 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T14:21:16.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_report_timeout": "900", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T14:21:16.264 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_osds": "4", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_interval": "86400", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_max_keys": "100", 2024-09-21T14:21:16.265 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_timeout": "300", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_session_timeout": "300", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_smart_report_timeout": "5", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_size": "4", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_debug": "false", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T14:21:16.266 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_tick_interval": "5", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T14:21:16.267 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_older_version": "true", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "monmap": "", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T14:21:16.268 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_tracing_enabled": "false", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_op_threads": "3", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_cm": "false", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_device_name": "", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T14:21:16.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_sl": "3", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_type": "ib", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_reap_threshold": "5", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_before_connect": "false", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv4": "true", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv6": "false", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr1": "true", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr2": "true", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_max": "7568", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_min": "6800", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_count": "3", 2024-09-21T14:21:16.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_delay": "5", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_client": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mds": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mgr": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mon": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_osd": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_client_mode": "crc secure", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_type": "", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_compress_secure": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_data": "true", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_header": "true", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T14:21:16.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bug": "true", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_old_message": "true", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hugepages": "", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_lro": "true", 2024-09-21T14:21:16.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_pmd": "", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_port_id": "0", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_on_send": "false", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_type": "", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_network_congestion": "0", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_accept_failures": "4", 2024-09-21T14:21:16.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_backoff": "15.000000", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_mode": "none", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_min_cost": "65536", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_public_type": "", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_service_mode": "crc secure", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_nodelay": "true", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T14:21:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_type": "async+posix", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "no_config_file": "false", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_ops": "1024", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_timeout": "10.000000", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "objectstore_blackhole": "false", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "openssl_engine_opts": "", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_ops": "4", 2024-09-21T14:21:16.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_slop": "0.020000", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_asio_thread_count": "2", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_max": "512", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_min": "64", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_peering": "false", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_beacon_report_interval": "300", 2024-09-21T14:21:16.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_duration": "30", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blkin_trace_all": "false", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_update_on_start": "true", 2024-09-21T14:21:16.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_cap": "256", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_op_priority": "63", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_watch_timeout": "30", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_command_max_records": "256", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_compact_on_start": "false", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_on_start": "true", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T14:21:16.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_op_order": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_shutdown": "false", 2024-09-21T14:21:16.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_notify_timeout": "30", 2024-09-21T14:21:16.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_enable_op_tracker": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_info": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T14:21:16.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_function_tracing": "false", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_grace": "20", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_interval": "6", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_stale": "600", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T14:21:16.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_size": "5120", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_kill_backfill_at": "0", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_cache_size": "50", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_dedup": "true", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max": "40", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_name_len": "100", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_size": "0", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_backfills": "1", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_count": "5", 2024-09-21T14:21:16.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_period": "600", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_name_len": "2048", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_size": "134217728", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pgls": "1024", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_cost": "8388608", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_objects": "10", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_scrubs": "3", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T14:21:16.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_size": "90", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_profile": "balanced", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T14:21:16.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_base": "805306368", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target": "4294967296", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_autotune": "false", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T14:21:16.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_recovery_priority": "0", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_interval": "5", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_cache_shards": "32", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_node": "-1", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objecter_finishers": "1", 2024-09-21T14:21:16.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore": "bluestore", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_fuse": "false", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_tracing": "false", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_duration": "600", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_size": "20", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_log_threshold": "5", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards": "0", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T14:21:16.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_timeout": "15", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_open_classes_on_start": "true", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_os_flags": "0", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_peering_op_priority": "255", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_priority": "5", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T14:21:16.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crimson": "false", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T14:21:16.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flags": "0", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_min_size": "0", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_size": "2", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_type": "replicated", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T14:21:16.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_cost": "20971520", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active": "0", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_priority": "3", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T14:21:16.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_priority": "5", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_repair_during_recovery": "false", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T14:21:16.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_cost": "52428800", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_hour": "0", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T14:21:16.294 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_priority": "5", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_skip_data_digest": "false", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_smart_report_timeout": "5", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_priority": "5", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T14:21:16.295 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_transaction_size": "30", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T14:21:16.296 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tracing": "false", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_use_stale_snap": "false", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_kill_at": "0", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_max_join_drift": "10", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min": "500", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min_wait": "0.050000", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max": "500", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T14:21:16.297 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_min": "250", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_stash_full_interval": "25", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_max": "500", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_min": "250", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "perf": "true", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "-", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrv": "", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "public_bind_addr": "-", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network": "", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network_interface": "", 2024-09-21T14:21:16.298 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_enabled": "false", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_mon_op_timeout": "0", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_osd_op_timeout": "0", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_tracing": "false", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_atime_update_interval": "60", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache": "true", 2024-09-21T14:21:16.299 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_policy": "writearound", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_size": "33554432", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_compression_hint": "none", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_clone_format": "auto", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_data_pool": "", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_features": "61", 2024-09-21T14:21:16.300 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_format": "2", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_map_options": "", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_order": "22", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_pool": "rbd", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_count": "0", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler": "simple", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T14:21:16.301 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_order": "24", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_pool": "", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_splay_width": "4", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T14:21:16.302 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T14:21:16.303 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T14:21:16.304 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_threads": "1", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_plugins": "", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T14:21:16.305 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T14:21:16.306 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_tracing": "false", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_names": "true", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_pool": "true", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_base_url": "", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_log_level": "", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_admin_entry": "admin", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_backend_store": "rados", 2024-09-21T14:21:16.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_barbican_url": "", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_beast_enable_async": "true", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_enabled": "true", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_lru_size": "10000", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_config_store": "rados", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_content_length_compat": "false", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T14:21:16.308 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cross_domain_policy": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T14:21:16.309 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T14:21:16.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T14:21:16.311 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_window": "30", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T14:21:16.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T14:21:16.313 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_name": "", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_s3website_name": "", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_ops_log": "false", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_static_website": "false", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_usage_log": "false", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_expose_bucket": "false", 2024-09-21T14:21:16.314 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_extended_http_attrs": "", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_filter": "none", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_objs": "32", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T14:21:16.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_init_timeout": "300", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_project": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_user": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_api_version": "2", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T14:21:16.316 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_url": "", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_objs": "32", 2024-09-21T14:21:16.317 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_worker": "3", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_http_headers": "", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T14:21:16.318 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_size": "0", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_listing_results": "1000", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_notify_retries": "10", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T14:21:16.319 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T14:21:16.320 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_control_oids": "8", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_numa_node": "-1", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T14:21:16.321 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_tracing": "false", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_token": "", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_url": "", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_rados": "false", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T14:21:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_continue": "true", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_tracing": "false", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm": "", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_id": "", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T14:21:16.323 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region": "", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_request_uri": "", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_resolve_cname": "false", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_run_sync_thread": "true", 2024-09-21T14:21:16.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_script_uri": "", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_service_provider_name": "", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_id": "", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_secret": "", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_entry": "sts", 2024-09-21T14:21:16.325 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_key": "sts", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_url": "", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_custom_header": "", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_need_stats": "true", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_tenant_name": "", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url": "", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T14:21:16.326 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_lease_period": "120", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_thread_pool_size": "512", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_comment": "", 2024-09-21T14:21:16.327 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_createby": "", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_encoding": "", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_flag": "false", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_origin": "", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_tracker": "", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_shards": "32", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_use_opa_authz": "false", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T14:21:16.328 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_unique_email": "true", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_verify_ssl": "true", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone": "", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_id": "", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup": "", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_id": "", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T14:21:16.329 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_block_size": "4096", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T14:21:16.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_partition_filters": "false", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_perf": "false", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "run_dir": "/var/run/ceph", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_block_create": "true", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T14:21:16.331 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_device_size": "53687091200", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_main_device_type": "SSD", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_segment_size": "67108864", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "set_keepcaps": "false", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "setgroup": "", 2024-09-21T14:21:16.332 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser": "", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser_match_path": "", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "thp": "false", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_default_timeout": "60", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout: "throttler_perf_counter": "true" 2024-09-21T14:21:16.333 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:16.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T14:21:16.450 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-09-21T14:21:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:21:16.460 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-09-21T14:21:16.460 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:21:16.468 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-09-21T14:21:16.469 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:21:16.479 DEBUG:teuthology.orchestra.run.smithi161:> type ceph-objectstore-tool 2024-09-21T14:21:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T14:21:16.497 DEBUG:teuthology.orchestra.run.smithi073:> type ceph-objectstore-tool 2024-09-21T14:21:16.512 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T14:21:16.512 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T14:21:16.522 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T14:21:16.522 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T14:21:16.522 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:16.523 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:21:16.523 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T14:21:16.523 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T14:21:16.523 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:16.524 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T14:21:16.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:16.524 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T14:21:16.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:21:16.525 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-21T14:21:16.554 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:16.553+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:16.592 INFO:teuthology.orchestra.run.smithi073.stdout:mapped writes DISABLED 2024-09-21T14:21:16.592 INFO:teuthology.orchestra.run.smithi073.stdout:mapped reads DISABLED 2024-09-21T14:21:16.592 INFO:teuthology.orchestra.run.smithi073.stdout:Seed set to 9185 2024-09-21T14:21:16.633 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:16.633+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:16.667 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:16.668 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:16.668 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:16.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:16.690 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:16.707 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:16.725 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:16.726 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:16.726 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:16.725+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:16.743 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:16.743 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:16.775 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:16.795 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:16.795 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:16.797 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:16.797 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:16.806 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:16.812 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:16.812 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:16.827 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:16.827+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:16.829 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:16.829 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:16.915 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.069 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:17.071 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:17.070+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:17.137 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:21:17.137 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:21:17.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T14:21:17.171 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:17.171+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:17.220 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.221 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:17.272 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:17.272+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:17.373 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:17.373+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:17.376 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.376 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:17.473 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:17.473+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:17.489 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.489 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:17.582 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:17.582+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:17.640 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.640 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.649 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:17.690 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:17.690+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:17.775 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:17.776+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:17.791 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.791 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:17.884 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:17.884+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:17.943 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:17.943 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:17.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:17.992 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:17.992+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:18.078 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:18.079+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:18.093 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.093 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T14:21:18.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T14:21:18.202 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:18.202+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:18.215 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.215 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:18.310 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:18.310+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:18.370 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.370 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:18.421 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:18.421+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:18.482 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:18.482+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:18.493 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.494 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.502 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:18.587 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:18.587+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:18.645 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.645 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.653 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:18.695 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:18.695+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:18.785 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:18.785+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:18.796 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.796 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.804 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:18.842 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:18.886 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:18.886+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:18.907 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:18.907 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:18.915 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:18.961 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:18.961 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:18.978 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:18.978 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:18.986 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:18.985+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:18.993 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:18.993 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:19.009 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:19.009 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:19.018 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:19.018 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:19.027 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:19.027 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:19.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:19.043 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:19.043 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:19.059 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:19.060 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:19.076 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:19.076 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:19.128 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:19.128+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:19.140 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T14:21:19.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T14:21:19.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:37 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'23, mlcod=16'15)", 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.768975+0000", 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.4033474810000008, 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0036986160000000001, 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.768975+0000", 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.768975+0000", 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.768975+0000", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.768981+0000", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2169999999999999e-06 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.768990+0000", 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3659999999999998e-06 2024-09-21T14:21:19.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.769005+0000", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4657e-05 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.769040+0000", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4974000000000001e-05 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.769055+0000", 2024-09-21T14:21:19.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4457999999999999e-05 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.772641+0000", 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0035863739999999998 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.772674+0000", 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3216000000000002e-05 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.809560+0000", 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3627629829999997, 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0038395349999999998, 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809560+0000", 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809560+0000", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809559+0000", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809565+0000", 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8189999999999997e-06 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809591+0000", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5922e-05 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809596+0000", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0939999999999998e-06 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809670+0000", 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4036999999999996e-05 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809686+0000", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.6028e-05 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.813372+0000", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0036857359999999998 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.813400+0000", 2024-09-21T14:21:19.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7458e-05 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.809996+0000", 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3623270359999999, 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0037315429999999999, 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809996+0000", 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809996+0000", 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.809995+0000", 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.810106+0000", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000110803 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.810115+0000", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4619999999999996e-06 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.810120+0000", 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.463e-06 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.810150+0000", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9944e-05 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.810160+0000", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0386e-05 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.813699+0000", 2024-09-21T14:21:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.003538729 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.813727+0000", 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8266e-05 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-21T14:21:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.830751+0000", 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3415718529999996, 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0052863509999999999, 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830751+0000", 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830751+0000", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830751+0000", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830857+0000", 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105979 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830865+0000", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1219999999999995e-06 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830869+0000", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830901+0000", 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1931999999999999e-05 2024-09-21T14:21:19.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.830914+0000", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2755000000000001e-05 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836004+0000", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0050906859999999996 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836037+0000", 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3268000000000001e-05 2024-09-21T14:21:19.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:85 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.831318+0000", 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3410046320000006, 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047343719999999997, 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831318+0000", 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831318+0000", 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831318+0000", 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831324+0000", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9800000000000003e-06 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831333+0000", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.6460000000000005e-06 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831339+0000", 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7640000000000002e-06 2024-09-21T14:21:19.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831371+0000", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1528e-05 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831390+0000", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9352e-05 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836039+0000", 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0046490780000000001 2024-09-21T14:21:19.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836053+0000", 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3762e-05 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'51)", 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.831789+0000", 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3405337979999992, 2024-09-21T14:21:19.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0042758620000000001, 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831789+0000", 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831789+0000", 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831789+0000", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831793+0000", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831800+0000", 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1389999999999998e-06 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831811+0000", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1224e-05 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831848+0000", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6940999999999997e-05 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.831862+0000", 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3937000000000001e-05 2024-09-21T14:21:19.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836053+0000", 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00419105 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.836065+0000", 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1715999999999999e-05 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.201 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:87 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'51)", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.832242+0000", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3400810700000001, 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0063077209999999996, 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832242+0000", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832242+0000", 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832241+0000", 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832264+0000", 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3039000000000001e-05 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832275+0000", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1137e-05 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832280+0000", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7360000000000001e-06 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832365+0000", 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5215000000000003e-05 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.832380+0000", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4897e-05 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.838520+0000", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0061402619999999996 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.838550+0000", 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9224e-05 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:88 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'58, mlcod=16'51)", 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.833266+0000", 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.3390572120000002, 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0053074530000000002, 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833266+0000", 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833266+0000", 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833265+0000", 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833271+0000", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4520000000000002e-06 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833282+0000", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0088000000000001e-05 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833289+0000", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6000000000000001e-06 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833320+0000", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1222e-05 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.833332+0000", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1868000000000001e-05 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.838551+0000", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0052189439999999997 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.838573+0000", 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2036e-05 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.952709+0000", 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2196143419999999, 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.003716741, 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952709+0000", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952709+0000", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952708+0000", 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952785+0000", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6365000000000005e-05 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952792+0000", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8720000000000002e-06 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952797+0000", 2024-09-21T14:21:19.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952833+0000", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5114999999999998e-05 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.952844+0000", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1065e-05 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956391+0000", 2024-09-21T14:21:19.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.003547747 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956425+0000", 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3995000000000002e-05 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.954208+0000", 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2181147840000008, 2024-09-21T14:21:19.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0045997, 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954208+0000", 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954208+0000", 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954208+0000", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954282+0000", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4250000000000002e-05 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954289+0000", 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8480000000000003e-06 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954293+0000", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3440000000000003e-06 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954321+0000", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7897000000000001e-05 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954331+0000", 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0125999999999999e-05 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958778+0000", 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0044471479999999997 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958808+0000", 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9436000000000001e-05 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.218 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:110 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.954744+0000", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2175794409999998, 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0040894900000000003, 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954744+0000", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954744+0000", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954743+0000", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954747+0000", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9589999999999997e-06 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954753+0000", 2024-09-21T14:21:19.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.954758+0000", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1999999999999996e-06 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955754+0000", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00099673100000000005 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955784+0000", 2024-09-21T14:21:19.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9485e-05 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958809+0000", 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.003025766 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958833+0000", 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3529000000000001e-05 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'66)", 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.955094+0000", 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2172294000000008, 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0037548859999999998, 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955094+0000", 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955094+0000", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955093+0000", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955098+0000", 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4519999999999996e-06 2024-09-21T14:21:19.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955108+0000", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.008e-05 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955114+0000", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7939999999999999e-06 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955964+0000", 2024-09-21T14:21:19.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00084956599999999995 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955977+0000", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3032e-05 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958834+0000", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.002856988 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.958848+0000", 2024-09-21T14:21:19.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4701e-05 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.955578+0000", 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2167447980000006, 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0059741020000000002, 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955578+0000", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955578+0000", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955576+0000", 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955722+0000", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00014562000000000001 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955752+0000", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0227e-05 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955778+0000", 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5384999999999998e-05 2024-09-21T14:21:19.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956141+0000", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00036340400000000001 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956153+0000", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1985e-05 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961526+0000", 2024-09-21T14:21:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0053728100000000004 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961552+0000", 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6390999999999999e-05 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'66)", 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.955820+0000", 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2165031319999997, 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00574634, 2024-09-21T14:21:19.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955820+0000", 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955820+0000", 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955819+0000", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955829+0000", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4590000000000004e-06 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955837+0000", 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.909e-06 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.955842+0000", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1279999999999999e-06 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956473+0000", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00063150200000000004 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956485+0000", 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1897e-05 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961554+0000", 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0050684809999999997 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961566+0000", 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2472e-05 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'66)", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.956720+0000", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2156029210000003, 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0048745050000000003, 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956720+0000", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956720+0000", 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956719+0000", 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956741+0000", 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1214000000000001e-05 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956748+0000", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4070000000000004e-06 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956753+0000", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7530000000000001e-06 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956783+0000", 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0207000000000001e-05 2024-09-21T14:21:19.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956842+0000", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9147000000000001e-05 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961567+0000", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047245200000000003 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961595+0000", 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7847999999999998e-05 2024-09-21T14:21:19.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:115 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'66)", 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.956760+0000", 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2155630110000004, 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0048476750000000001, 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956760+0000", 2024-09-21T14:21:19.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956760+0000", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956760+0000", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956965+0000", 2024-09-21T14:21:19.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00020505400000000001 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956973+0000", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9209999999999992e-06 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.956978+0000", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8080000000000004e-06 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957080+0000", 2024-09-21T14:21:19.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010265 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957099+0000", 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.91e-05 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961595+0000", 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0044960750000000004 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961608+0000", 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2306e-05 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:116 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'78, mlcod=16'66)", 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.957075+0000", 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2152483650000008, 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00454727, 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957075+0000", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957075+0000", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957074+0000", 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957077+0000", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.582e-06 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957081+0000", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2880000000000003e-06 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957084+0000", 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8770000000000002e-06 2024-09-21T14:21:19.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957361+0000", 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000277423 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957373+0000", 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1542999999999999e-05 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961609+0000", 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0042363050000000001 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961622+0000", 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2563e-05 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:117 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'69)", 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:09.957659+0000", 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.2146638070000009, 2024-09-21T14:21:19.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0039763619999999998, 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957659+0000", 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957659+0000", 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957659+0000", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957662+0000", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.49e-06 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957668+0000", 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.468e-06 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957671+0000", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0419999999999998e-06 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957701+0000", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0173000000000001e-05 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.957712+0000", 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1314999999999999e-05 2024-09-21T14:21:19.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961623+0000", 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0039102579999999998 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:09.961636+0000", 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2979000000000001e-05 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:139 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'86)", 2024-09-21T14:21:19.252 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:10.288531+0000", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.8837915439999993, 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0040975339999999999, 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288531+0000", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288531+0000", 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288531+0000", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288535+0000", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288541+0000", 2024-09-21T14:21:19.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8300000000000001e-06 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288547+0000", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3029999999999996e-06 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288574+0000", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7186999999999999e-05 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288584+0000", 2024-09-21T14:21:19.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0033e-05 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.292600+0000", 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0040158069999999997 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.292629+0000", 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8979000000000002e-05 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.256 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4172.0:140 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'86)", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:10.288952+0000", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.8833706370000005, 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.003692163, 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288952+0000", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288952+0000", 2024-09-21T14:21:19.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.288952+0000", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.289011+0000", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8761999999999997e-05 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.289019+0000", 2024-09-21T14:21:19.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2169999999999994e-06 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.289023+0000", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8489999999999999e-06 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.289052+0000", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9342000000000001e-05 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.289063+0000", 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0910000000000001e-05 2024-09-21T14:21:19.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.292630+0000", 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0035674000000000001 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:10.292644+0000", 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4163e-05 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:19.260 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:19.263 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:19.261+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:19.362 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:19.361+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:19.407 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.407 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:19.407 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:19.407 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:19.500 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:19.499+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:19.558 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:19.558 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.564 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:19.563+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:19.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:19.665 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:19.665+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:19.709 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.709 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:19.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:19.709 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [] 2024-09-21T14:21:19.709 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.718 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:19.767 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:19.766+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:19.820 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.820 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:19.820 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:19.820 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.828 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:19.867 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:19.867+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:19.929 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:19.929 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:19.929 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:19.929 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:19.929 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:19.937 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:19.981 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:19.980+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:20.040 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:20.040 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:20.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:20.040 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [] 2024-09-21T14:21:20.040 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:20.048 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:20.093 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:20.093+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:20.143 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:21:20.143 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T14:21:20.159 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:20.159 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:20.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:21:20.169 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:20.168+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:20.170 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:20.269 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:20.269+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:20.324 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.324 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:20.325 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:20.325 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:20.325 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:20.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:20.371 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:20.371+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:20.439 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.439 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:20.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15)", 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.896899524, 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015540700000000001, 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:20.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541786+0000", 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541792+0000", 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8830000000000003e-06 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541800+0000", 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1969999999999996e-06 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541807+0000", 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9149999999999999e-06 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541854+0000", 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6987999999999998e-05 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541943+0000", 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8875000000000005e-05 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:3 1.2 1:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15)", 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:08.542092+0000", 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.896595810000001, 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110021, 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:20.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542092+0000", 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542092+0000", 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542091+0000", 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542095+0000", 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7260000000000002e-06 2024-09-21T14:21:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542104+0000", 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4950000000000008e-06 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542109+0000", 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9479999999999999e-06 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542142+0000", 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3541999999999998e-05 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.542202+0000", 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9061000000000003e-05 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.903064235, 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027325829999999998, 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 5 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535621+0000", 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535635+0000", 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3899e-05 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535653+0000", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8423999999999998e-05 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535665+0000", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1993e-05 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535851+0000", 2024-09-21T14:21:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186119 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535967+0000", 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115701 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.536099+0000", 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000132022 2024-09-21T14:21:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537334+0000", 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001234684 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538262+0000", 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092820100000000003 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538316+0000", 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4540000000000003e-05 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538356+0000", 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9329999999999998e-05 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4254.0:5 1.2 e15/14 ondisk, result = 0)", 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.537260+0000", 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.901427431, 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6768999999999996e-05, 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537260+0000", 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537260+0000", 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537259+0000", 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537263+0000", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.498e-06 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537278+0000", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4299e-05 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537285+0000", 2024-09-21T14:21:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.793e-06 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537322+0000", 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7437000000000001e-05 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537329+0000", 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1860000000000003e-06 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537337+0000", 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4930000000000001e-06 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.7703931970000006, 2024-09-21T14:21:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057806730000000001, 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668290+0000", 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999957 2024-09-21T14:21:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668307+0000", 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7039000000000001e-05 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668323+0000", 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6753e-05 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668336+0000", 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2576e-05 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668402+0000", 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5927999999999996e-05 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668411+0000", 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3049999999999998e-06 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.672915+0000", 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004503295 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.672978+0000", 2024-09-21T14:21:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3864999999999999e-05 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673080+0000", 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101183 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673719+0000", 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063918899999999997 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673995+0000", 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00027601299999999997 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674038+0000", 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3214999999999999e-05 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674075+0000", 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6806e-05 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4254.0:7 1.2 e17/14 ondisk, result = 0)", 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.7647639650000002, 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000158071, 2024-09-21T14:21:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673922+0000", 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673927+0000", 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7180000000000004e-06 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673937+0000", 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6870000000000006e-06 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673943+0000", 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0719999999999996e-06 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673974+0000", 2024-09-21T14:21:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1239999999999999e-05 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673980+0000", 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.657e-06 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674081+0000", 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100925 2024-09-21T14:21:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:10 3.1c e22/19 3:3831950c:::rbd_directory:head v 22'1, mlcod=0'0)", 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.28946907599999999, 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093686500000000005, 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149223+0000", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149239+0000", 2024-09-21T14:21:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6290000000000002e-05 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149250+0000", 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0098000000000001e-05 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149312+0000", 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2656000000000005e-05 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149333+0000", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0996999999999999e-05 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150127+0000", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000794255 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150155+0000", 2024-09-21T14:21:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7654999999999999e-05 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:12 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.28243247199999999, 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014151999999999999, 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 12 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156253+0000", 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156264+0000", 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0881e-05 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156281+0000", 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6929e-05 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156289+0000", 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2979999999999992e-06 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156346+0000", 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6709999999999997e-05 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156504+0000", 2024-09-21T14:21:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000157999 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 1", 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156759+0000", 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025506200000000002 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157433+0000", 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067386200000000001 2024-09-21T14:21:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157573+0000", 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014004100000000001 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157628+0000", 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5086000000000001e-05 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157670+0000", 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2205e-05 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:12 3.f e22/19 ondisk, result = 0)", 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.28118556099999997, 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017642800000000001, 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157501+0000", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157506+0000", 2024-09-21T14:21:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7600000000000002e-06 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157515+0000", 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6020000000000006e-06 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157522+0000", 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6980000000000001e-06 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157557+0000", 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213000000000003e-05 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157566+0000", 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0319999999999992e-06 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157678+0000", 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000111877 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:16 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.27011292799999997, 2024-09-21T14:21:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032511399999999999, 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 16 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168572+0000", 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168585+0000", 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.322e-05 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168607+0000", 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2042999999999999e-05 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168627+0000", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9241000000000001e-05 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168671+0000", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4674e-05 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168743+0000", 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1156999999999997e-05 2024-09-21T14:21:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168899+0000", 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015698400000000001 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:17 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26968957599999999, 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017967100000000001, 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 17 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168997+0000", 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169002+0000", 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2349999999999999e-06 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169010+0000", 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0509999999999997e-06 2024-09-21T14:21:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169015+0000", 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169043+0000", 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8243e-05 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169099+0000", 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6326999999999998e-05 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169177+0000", 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8102999999999994e-05 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:18 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26942796899999999, 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000174036, 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 18 2024-09-21T14:21:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169263+0000", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169271+0000", 2024-09-21T14:21:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3250000000000008e-06 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169276+0000", 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1640000000000003e-06 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169298+0000", 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2829000000000001e-05 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169342+0000", 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3488000000000001e-05 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169433+0000", 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1500000000000001e-05 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:19 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.269152636, 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000146105, 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 19 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169534+0000", 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169539+0000", 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9099999999999996e-06 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169549+0000", 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0597999999999999e-05 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169556+0000", 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.046e-06 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169581+0000", 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4454e-05 2024-09-21T14:21:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169620+0000", 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9397999999999998e-05 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169681+0000", 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0547999999999997e-05 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:20 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.268592106, 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031777800000000001, 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 20 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170104+0000", 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5079999999999994e-06 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170113+0000", 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0480000000000004e-06 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170119+0000", 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3550000000000001e-06 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170152+0000", 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2268999999999999e-05 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170202+0000", 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0639000000000001e-05 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170413+0000", 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021063200000000001 2024-09-21T14:21:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26816604799999999, 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291587, 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170525+0000", 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3120000000000004e-06 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170533+0000", 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0369999999999995e-06 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170539+0000", 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170564+0000", 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5480999999999999e-05 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170625+0000", 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1007000000000001e-05 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170813+0000", 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018774899999999999 2024-09-21T14:21:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:22 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26778054800000001, 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023776599999999999, 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 22 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170906+0000", 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170910+0000", 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1350000000000002e-06 2024-09-21T14:21:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170921+0000", 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0974e-05 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170927+0000", 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7939999999999999e-06 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170956+0000", 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8755000000000002e-05 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171028+0000", 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2043000000000001e-05 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171145+0000", 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011680600000000001 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:24 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26706685899999999, 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019065999999999999, 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 24 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171625+0000", 2024-09-21T14:21:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6400000000000002e-06 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171636+0000", 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0628999999999999e-05 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171650+0000", 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3964e-05 2024-09-21T14:21:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171682+0000", 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2014999999999998e-05 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171732+0000", 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0204999999999998e-05 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171811+0000", 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9083000000000002e-05 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:25 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [watch watch cookie 140451940236640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26672041000000002, 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0009969, 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 25 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171966+0000", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171971+0000", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9559999999999996e-06 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171981+0000", 2024-09-21T14:21:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0182e-05 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172029+0000", 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7817e-05 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172060+0000", 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0568000000000003e-05 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172104+0000", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4295000000000002e-05 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 1", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172198+0000", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4034000000000001e-05 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172715+0000", 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051702900000000001 2024-09-21T14:21:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172900+0000", 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018541599999999999 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172927+0000", 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6409000000000001e-05 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172964+0000", 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6971999999999998e-05 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:25 3.f e22/19 ondisk, result = 0)", 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.26585282700000001, 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013298800000000001, 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172839+0000", 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1000000000000003e-06 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172850+0000", 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1260999999999999e-05 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172864+0000", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4121000000000001e-05 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172890+0000", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5783999999999999e-05 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172897+0000", 2024-09-21T14:21:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2470000000000002e-06 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172967+0000", 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0358000000000006e-05 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:20.543 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:20.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:20.545 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:20.545+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:20.645 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:20.646+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:20.688 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:20.688 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:20.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:20.781 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:20.781+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:20.842 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.842 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:20.842 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:20.842 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:20.842 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:20.848 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:20.847+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:20.851 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:20.948 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:20.947+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4254.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.459259696, 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018998559999999999, 2024-09-21T14:21:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536834+0000", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2065000000000001e-05 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536845+0000", 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1073000000000001e-05 2024-09-21T14:21:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536853+0000", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8029999999999997e-06 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536895+0000", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2753999999999998e-05 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536917+0000", 2024-09-21T14:21:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.192e-05 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.538696+0000", 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001778473 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.538722+0000", 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6336999999999999e-05 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.8530424490000001, 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029151810000000002, 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143037+0000", 2024-09-21T14:21:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143052+0000", 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.449e-05 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143070+0000", 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8105e-05 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143082+0000", 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2045000000000001e-05 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143128+0000", 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6109999999999997e-05 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143133+0000", 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5449999999999997e-06 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144568+0000", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001435497 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144692+0000", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012378700000000001 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144830+0000", 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138042 2024-09-21T14:21:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145538+0000", 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00070826400000000001 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145872+0000", 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033409800000000001 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145918+0000", 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5388e-05 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145955+0000", 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6874e-05 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:7 3.1a e21/19 ondisk, result = 0)", 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:19.145805+0000", 2024-09-21T14:21:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.8502770630000001, 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015859500000000001, 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145805+0000", 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145805+0000", 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145804+0000", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145811+0000", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6810000000000001e-06 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145820+0000", 2024-09-21T14:21:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0489999999999993e-06 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145827+0000", 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.81e-06 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145857+0000", 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0040999999999999e-05 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145865+0000", 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5650000000000002e-06 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145963+0000", 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8208000000000006e-05 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.85157221400000005, 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039806640000000001, 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144507+0000", 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144519+0000", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1284000000000001e-05 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144535+0000", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144545+0000", 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9110000000000005e-06 2024-09-21T14:21:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144595+0000", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0318999999999997e-05 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144697+0000", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010137999999999999 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144844+0000", 2024-09-21T14:21:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014769800000000001 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.147490+0000", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026452289999999998 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148368+0000", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000878738 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148425+0000", 2024-09-21T14:21:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7167999999999999e-05 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148490+0000", 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4828999999999999e-05 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:9 3.1a e21/19 ondisk, result = 0)", 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.148271+0000", 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.84781070300000005, 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022998199999999999, 2024-09-21T14:21:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148271+0000", 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148271+0000", 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148270+0000", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148281+0000", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0721e-05 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148297+0000", 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6140000000000001e-05 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148307+0000", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0502e-05 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148349+0000", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2038e-05 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148361+0000", 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1522000000000001e-05 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148501+0000", 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014021199999999999 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.84744683799999998, 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048317120000000002, 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148634+0000", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148643+0000", 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8289999999999997e-06 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148652+0000", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.312e-06 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148659+0000", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9800000000000001e-06 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148708+0000", 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9147999999999998e-05 2024-09-21T14:21:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148852+0000", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014422 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149086+0000", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023317999999999999 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150494+0000", 2024-09-21T14:21:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001408402 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153356+0000", 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028617360000000001 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153421+0000", 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5625999999999998e-05 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153467+0000", 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5402999999999999e-05 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:10 3.1c e21/19 ondisk, result = 0)", 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.150308+0000", 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.84577343800000004, 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000188994, 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150308+0000", 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150308+0000", 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150308+0000", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150313+0000", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3059999999999997e-06 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150327+0000", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.366e-05 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150336+0000", 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2529999999999993e-06 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150478+0000", 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000142459 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150488+0000", 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3940000000000002e-06 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150497+0000", 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.611e-06 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:11 3.1f e22/19 3:fa654f18:::rbd_id.image_client.0:head v 22'1, mlcod=0'0)", 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.84207097799999997, 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018756840000000001, 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154010+0000", 2024-09-21T14:21:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154019+0000", 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7590000000000003e-06 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154029+0000", 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0546e-05 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154045+0000", 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5739e-05 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154081+0000", 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5806000000000002e-05 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154098+0000", 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7717e-05 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.155838+0000", 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001739263 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.155887+0000", 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8807000000000002e-05 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:12 3.f e22/19 3:f0449c44:::rbd_header.1121746b0c91:head v 22'1, mlcod=0'0)", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.156831+0000", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.83925100200000002, 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063504499999999999, 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156831+0000", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156831+0000", 2024-09-21T14:21:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156830+0000", 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156839+0000", 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9849999999999995e-06 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156852+0000", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2935000000000001e-05 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156859+0000", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0910000000000004e-06 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156895+0000", 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6059999999999997e-05 2024-09-21T14:21:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156916+0000", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0941000000000002e-05 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157429+0000", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051335699999999998 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157466+0000", 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6767000000000002e-05 2024-09-21T14:21:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.171236+0000", 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.82484537499999999, 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020237200000000001, 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 23 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171236+0000", 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171236+0000", 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171234+0000", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171243+0000", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171259+0000", 2024-09-21T14:21:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.552e-05 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171267+0000", 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9179999999999994e-06 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171315+0000", 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8161000000000002e-05 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171372+0000", 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7145000000000003e-05 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171439+0000", 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6971999999999995e-05 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:25 3.f e22/19 3:f0449c44:::rbd_header.1121746b0c91:head v 22'2, mlcod=0'0)", 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.172268+0000", 2024-09-21T14:21:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.82381428700000003, 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00052761699999999997, 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172268+0000", 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172268+0000", 2024-09-21T14:21:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172266+0000", 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172275+0000", 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999994e-06 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172290+0000", 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4307e-05 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172297+0000", 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7710000000000001e-06 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172341+0000", 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4166999999999999e-05 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172366+0000", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4808999999999999e-05 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172765+0000", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039918199999999999 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172795+0000", 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0454e-05 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:21.049 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:21.050 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:21.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:21.050+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:21.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:21.151 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:21.150+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:21.151 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:21:21.159 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:21.159 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:21.160 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:21.160 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:21.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:21:21.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:21.219 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:21.220 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:21.238 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:21.239 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:21.252 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:21.252+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:21.255 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:21.255 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:21.288 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:21.288 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:21.305 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:21.305 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:21.336 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:21.336 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:21.351 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.351 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.352 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:21.352+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:21.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:21.366 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:21.366 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:21.383 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:21.383 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:21.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:21.452+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:21.470 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.470 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:21.553 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:21.553+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:21.582 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.582 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:21.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:21.654+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:21.695 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.695 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:21.755 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:21.754+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:21.807 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.807 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.816 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:21.856 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:21.856+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:21.900 INFO:teuthology.orchestra.run.smithi073.stdout:1 write 0x4e4eb1b thru 0x4e5c5ac (0xda92 bytes) 2024-09-21T14:21:21.903 INFO:teuthology.orchestra.run.smithi073.stdout:2 writesame 0x3d40804 thru 0x3d49311 data_size 0x33(0x8b0e bytes) 2024-09-21T14:21:21.909 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xbf5640a 2024-09-21T14:21:21.917 INFO:teuthology.orchestra.run.smithi073.stdout:3 trunc from 0x4e5c5ad to 0xbf5640a 2024-09-21T14:21:21.922 INFO:teuthology.orchestra.run.smithi073.stdout:4 writesame 0xaa661df thru 0xaa6934e data_size 0x38(0x3170 bytes) 2024-09-21T14:21:21.928 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:21.928 INFO:teuthology.orchestra.run.smithi073.stdout:6 read 0x2b77310 thru 0x2b8090b (0x95fc bytes) 2024-09-21T14:21:21.934 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:21.935 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:21.937 INFO:teuthology.orchestra.run.smithi073.stdout:7 write 0xed74654 thru 0xed7a8c7 (0x6274 bytes) 2024-09-21T14:21:21.940 INFO:teuthology.orchestra.run.smithi073.stdout:8 write 0x7ccc5e thru 0x7cf6b4 (0x2a57 bytes) 2024-09-21T14:21:21.944 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:21.944 INFO:teuthology.orchestra.run.smithi073.stdout:9 write 0x2dc5e58 thru 0x2dd03e1 (0xa58a bytes) 2024-09-21T14:21:21.946 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:21.954 INFO:teuthology.orchestra.run.smithi073.stdout:11 trunc from 0xed7a8c8 to 0x8029d84 2024-09-21T14:21:21.956 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:21.956+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:21.963 INFO:teuthology.orchestra.run.smithi073.stdout:12 read 0x41cf252 thru 0x41d72e2 (0x8091 bytes) 2024-09-21T14:21:21.963 INFO:teuthology.orchestra.run.smithi073.stdout:13 write 0x4725b54 thru 0x472694c (0xdf9 bytes) 2024-09-21T14:21:21.971 INFO:teuthology.orchestra.run.smithi073.stdout:15 trunc from 0x8029d84 to 0x3fbbbc8 2024-09-21T14:21:21.994 INFO:teuthology.orchestra.run.smithi073.stdout:18 write 0x90df27a thru 0x90eac4c (0xb9d3 bytes) 2024-09-21T14:21:21.998 INFO:teuthology.orchestra.run.smithi073.stdout:21 writesame 0x792f341 thru 0x793d4da data_size 0x43(0xe19a bytes) 2024-09-21T14:21:22.001 INFO:teuthology.orchestra.run.smithi073.stdout:24 punch from 0x880f854 to 0x880f91c, (0xc8 bytes) 2024-09-21T14:21:22.001 INFO:teuthology.orchestra.run.smithi073.stdout:25 read 0x840f19a thru 0x8411cd7 (0x2b3e bytes) 2024-09-21T14:21:22.001 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.001 INFO:teuthology.orchestra.run.smithi073.stdout:31 writesame 0x1bd2bcb thru 0x1bde6f8 data_size 0x61(0xbb2e bytes) 2024-09-21T14:21:22.011 INFO:teuthology.orchestra.run.smithi073.stdout:32 trunc from 0x90eac4d to 0x2d84f0c 2024-09-21T14:21:22.021 INFO:teuthology.orchestra.run.smithi073.stdout:34 read 0x85cd13 thru 0x860b10 (0x3dfe bytes) 2024-09-21T14:21:22.030 INFO:teuthology.orchestra.run.smithi073.stdout:35 write 0x63073f0 thru 0x630efb5 (0x7bc6 bytes) 2024-09-21T14:21:22.035 INFO:teuthology.orchestra.run.smithi073.stdout:38 writesame 0xc80fe2 thru 0xc8566a data_size 0xd(0x4689 bytes) 2024-09-21T14:21:22.038 INFO:teuthology.orchestra.run.smithi073.stdout:40 clone 1 order 25 su 2097152 sc 7 2024-09-21T14:21:22.052 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.052 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.058 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:22.058+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:22.061 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:22.159 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:21:22.159 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:22.158+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:22.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:21:22.164 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.164 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:22.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:21:22.260 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:22.260+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:22.286 INFO:teuthology.orchestra.run.smithi073.stdout:42 trunc from 0x630efb6 to 0x601f113 2024-09-21T14:21:22.305 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.305 INFO:teuthology.orchestra.run.smithi073.stdout:44 write 0x52b2d52 thru 0x52c0add (0xdd8c bytes) 2024-09-21T14:21:22.307 INFO:teuthology.orchestra.run.smithi073.stdout:46 punch from 0x138fd82 to 0x13971c2, (0x7440 bytes) 2024-09-21T14:21:22.307 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.312 INFO:teuthology.orchestra.run.smithi073.stdout:48 write 0x69636ec thru 0x696b3ed (0x7d02 bytes) 2024-09-21T14:21:22.321 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.321 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.330 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:22.333 INFO:teuthology.orchestra.run.smithi073.stdout:49 write 0x94f6e85 thru 0x94fa482 (0x35fe bytes) 2024-09-21T14:21:22.337 INFO:teuthology.orchestra.run.smithi073.stdout:50 read 0x8bc0c37 thru 0x8bc4f73 (0x433d bytes) 2024-09-21T14:21:22.337 INFO:teuthology.orchestra.run.smithi073.stdout:51 read 0x4d00eff thru 0x4d08e4f (0x7f51 bytes) 2024-09-21T14:21:22.338 INFO:teuthology.orchestra.run.smithi073.stdout:52 punch from 0x75bc62e to 0x75bf1f7, (0x2bc9 bytes) 2024-09-21T14:21:22.338 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.338 INFO:teuthology.orchestra.run.smithi073.stdout:58 writesame 0x62809e5 thru 0x6281289 data_size 0x1(0x8a5 bytes) 2024-09-21T14:21:22.340 INFO:teuthology.orchestra.run.smithi073.stdout:59 read 0x1ca1767 thru 0x1ca71e4 (0x5a7e bytes) 2024-09-21T14:21:22.340 INFO:teuthology.orchestra.run.smithi073.stdout:61 write 0x3fa063c thru 0x3fa646e (0x5e33 bytes) 2024-09-21T14:21:22.342 INFO:teuthology.orchestra.run.smithi073.stdout:64 read 0x216946b thru 0x21721ec (0x8d82 bytes) 2024-09-21T14:21:22.342 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.342 INFO:teuthology.orchestra.run.smithi073.stdout:67 read 0x3eef462 thru 0x3eefd09 (0x8a8 bytes) 2024-09-21T14:21:22.342 INFO:teuthology.orchestra.run.smithi073.stdout:68 write 0x2a3f014 thru 0x2a45d2d (0x6d1a bytes) 2024-09-21T14:21:22.343 INFO:teuthology.orchestra.run.smithi073.stdout:69 write 0x668b5de thru 0x668dfe3 (0x2a06 bytes) 2024-09-21T14:21:22.347 INFO:teuthology.orchestra.run.smithi073.stdout:70 read 0x523e49 thru 0x533219 (0xf3d1 bytes) 2024-09-21T14:21:22.347 INFO:teuthology.orchestra.run.smithi073.stdout:71 writesame 0x7c5b907 thru 0x7c62824 data_size 0xb(0x6f1e bytes) 2024-09-21T14:21:22.351 INFO:teuthology.orchestra.run.smithi073.stdout:72 trunc from 0x94fa483 to 0x7138595 2024-09-21T14:21:22.364 INFO:teuthology.orchestra.run.smithi073.stdout:73 read 0x1db4199 thru 0x1dbf3b5 (0xb21d bytes) 2024-09-21T14:21:22.374 INFO:teuthology.orchestra.run.smithi073.stdout:74 write 0xb0e06f1 thru 0xb0ed1ed (0xcafd bytes) 2024-09-21T14:21:22.374 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:22.374+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:22.376 INFO:teuthology.orchestra.run.smithi073.stdout:76 read 0x6283157 thru 0x628794c (0x47f6 bytes) 2024-09-21T14:21:22.376 INFO:teuthology.orchestra.run.smithi073.stdout:77 read 0x731ea05 thru 0x732b711 (0xcd0d bytes) 2024-09-21T14:21:22.377 INFO:teuthology.orchestra.run.smithi073.stdout:78 trunc from 0xb0ed1ee to 0xb8954c6 2024-09-21T14:21:22.382 INFO:teuthology.orchestra.run.smithi073.stdout:80 writesame 0x6a429df thru 0x6a4c793 data_size 0x1(0x9db5 bytes) 2024-09-21T14:21:22.401 INFO:teuthology.orchestra.run.smithi073.stdout:81 writesame 0x2d233c0 thru 0x2d2f839 data_size 0x65(0xc47a bytes) 2024-09-21T14:21:22.404 INFO:teuthology.orchestra.run.smithi073.stdout:83 read 0x5ce78ad thru 0x5cf6be7 (0xf33b bytes) 2024-09-21T14:21:22.404 INFO:teuthology.orchestra.run.smithi073.stdout:84 read 0x397e845 thru 0x398ca33 (0xe1ef bytes) 2024-09-21T14:21:22.404 INFO:teuthology.orchestra.run.smithi073.stdout:85 write 0x87b9c5e thru 0x87beaef (0x4e92 bytes) 2024-09-21T14:21:22.406 INFO:teuthology.orchestra.run.smithi073.stdout:86 write 0x5e8638e thru 0x5e92f77 (0xcbea bytes) 2024-09-21T14:21:22.413 INFO:teuthology.orchestra.run.smithi073.stdout:87 trunc from 0xb8954c6 to 0x7329312 2024-09-21T14:21:22.424 INFO:teuthology.orchestra.run.smithi073.stdout:88 write 0x4639945 thru 0x463a6c5 (0xd81 bytes) 2024-09-21T14:21:22.425 INFO:teuthology.orchestra.run.smithi073.stdout:89 read 0x32b281f thru 0x32b53f8 (0x2bda bytes) 2024-09-21T14:21:22.430 INFO:teuthology.orchestra.run.smithi073.stdout:91 trunc from 0x7329312 to 0x2d9e734 2024-09-21T14:21:22.438 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.439 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.448 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:22.461 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:22.461+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:22.462 INFO:teuthology.orchestra.run.smithi073.stdout:92 write 0xdafaa77 thru 0xdb0663b (0xbbc5 bytes) 2024-09-21T14:21:22.464 INFO:teuthology.orchestra.run.smithi073.stdout:95 punch from 0x767755f to 0x767982e, (0x22cf bytes) 2024-09-21T14:21:22.464 INFO:teuthology.orchestra.run.smithi073.stdout:96 write 0xab19863 thru 0xab22c91 (0x942f bytes) 2024-09-21T14:21:22.465 INFO:teuthology.orchestra.run.smithi073.stdout:98 read 0x15b79bb thru 0x15c55e8 (0xdc2e bytes) 2024-09-21T14:21:22.477 INFO:teuthology.orchestra.run.smithi073.stdout:100 trunc from 0xdb0663c to 0x4e2d0a7 2024-09-21T14:21:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:101 punch from 0x5edd3c to 0x5f9973, (0xbc37 bytes) 2024-09-21T14:21:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:103 write 0x2614986 thru 0x262273c (0xddb7 bytes) 2024-09-21T14:21:22.489 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.491 INFO:teuthology.orchestra.run.smithi073.stdout:107 trunc from 0x4e2d0a7 to 0x6a783f7 2024-09-21T14:21:22.496 INFO:teuthology.orchestra.run.smithi073.stdout:108 read 0x4d8ec52 thru 0x4d9e78c (0xfb3b bytes) 2024-09-21T14:21:22.510 INFO:teuthology.orchestra.run.smithi073.stdout:112 write 0xd13e03e thru 0xd147a64 (0x9a27 bytes) 2024-09-21T14:21:22.511 INFO:teuthology.orchestra.run.smithi073.stdout:113 read 0x89e85a5 thru 0x89f5c62 (0xd6be bytes) 2024-09-21T14:21:22.512 INFO:teuthology.orchestra.run.smithi073.stdout:115 punch from 0x1f11c39 to 0x1f17a6c, (0x5e33 bytes) 2024-09-21T14:21:22.512 INFO:teuthology.orchestra.run.smithi073.stdout:116 punch from 0x7a0a11d to 0x7a162fb, (0xc1de bytes) 2024-09-21T14:21:22.512 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.512 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xd4097aa 2024-09-21T14:21:22.512 INFO:teuthology.orchestra.run.smithi073.stdout:119 trunc from 0xd147a65 to 0xd4097aa 2024-09-21T14:21:22.517 INFO:teuthology.orchestra.run.smithi073.stdout:120 writesame 0x3e23e2d thru 0x3e32f52 data_size 0x3(0xf126 bytes) 2024-09-21T14:21:22.527 INFO:teuthology.orchestra.run.smithi073.stdout:121 read 0x21a97da thru 0x21ad73e (0x3f65 bytes) 2024-09-21T14:21:22.527 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xe74215a 2024-09-21T14:21:22.529 INFO:teuthology.orchestra.run.smithi073.stdout:122 trunc from 0xd4097aa to 0xe74215a 2024-09-21T14:21:22.534 INFO:teuthology.orchestra.run.smithi073.stdout:123 writesame 0x40a714e thru 0x40a786f data_size 0xb(0x722 bytes) 2024-09-21T14:21:22.535 INFO:teuthology.orchestra.run.smithi073.stdout:125 read 0x1c4fd5d thru 0x1c5e36e (0xe612 bytes) 2024-09-21T14:21:22.535 INFO:teuthology.orchestra.run.smithi073.stdout:127 read 0xd72c432 thru 0xd73933e (0xcf0d bytes) 2024-09-21T14:21:22.536 INFO:teuthology.orchestra.run.smithi073.stdout:128 read 0x9e3daf0 thru 0x9e3e44f (0x960 bytes) 2024-09-21T14:21:22.536 INFO:teuthology.orchestra.run.smithi073.stdout:129 read 0xa4c7fe4 thru 0xa4d5a45 (0xda62 bytes) 2024-09-21T14:21:22.537 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.537 INFO:teuthology.orchestra.run.smithi073.stdout:131 write 0x6565d0e thru 0x6573412 (0xd705 bytes) 2024-09-21T14:21:22.538 INFO:teuthology.orchestra.run.smithi073.stdout:132 read 0xbe77032 thru 0xbe7aab4 (0x3a83 bytes) 2024-09-21T14:21:22.539 INFO:teuthology.orchestra.run.smithi073.stdout:133 writesame 0x873906d thru 0x87459bd data_size 0x29(0xc951 bytes) 2024-09-21T14:21:22.541 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:22.541 INFO:teuthology.orchestra.run.smithi073.stdout:136 write 0x29e3b07 thru 0x29f1b71 (0xe06b bytes) 2024-09-21T14:21:22.542 INFO:teuthology.orchestra.run.smithi073.stdout:138 write 0xdffe536 thru 0xe006b41 (0x860c bytes) 2024-09-21T14:21:22.546 INFO:teuthology.orchestra.run.smithi073.stdout:139 read 0xb99709e thru 0xb9a4de6 (0xdd49 bytes) 2024-09-21T14:21:22.562 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:22.561+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:22.563 INFO:teuthology.orchestra.run.smithi073.stdout:141 trunc from 0xe74215a to 0x392f50 2024-09-21T14:21:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:22.609 INFO:teuthology.orchestra.run.smithi073.stdout:142 write 0xd6187eb thru 0xd620d4b (0x8561 bytes) 2024-09-21T14:21:22.612 INFO:teuthology.orchestra.run.smithi073.stdout:144 write 0x6922e8 thru 0x69a65e (0x8377 bytes) 2024-09-21T14:21:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:146 writesame 0x532f2b7 thru 0x5336b20 data_size 0x1(0x786a bytes) 2024-09-21T14:21:22.630 INFO:teuthology.orchestra.run.smithi073.stdout:147 clone 2 order 23 su 65536 sc 5 2024-09-21T14:21:22.687 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:22.686+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:22.746 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.746 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.754 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:22.763 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:22.763+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:22.864 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:22.864+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:22.898 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:22.899 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:22.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:22.965 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:22.965+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:23.010 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:23.010 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:23.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:23.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:23.065+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:23.120 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:23.121 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:23.129 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:21:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:21:23.167 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:23.167+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:23.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:21:23.224 INFO:teuthology.orchestra.run.smithi073.stdout:flattening image image_client.0-clone1 2024-09-21T14:21:23.268 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:23.267+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:23.275 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:23.275 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:23.284 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:23.368 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T14:21:23.369 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:23.368+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:23.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:23.427 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:23.428 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:23.436 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:23.478 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:23.477+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:23.481 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:23.481 INFO:teuthology.orchestra.run.smithi073.stdout:149 punch from 0x5f25db to 0x5f2b09, (0x52e bytes) 2024-09-21T14:21:23.494 INFO:teuthology.orchestra.run.smithi073.stdout:150 trunc from 0xd620d4c to 0x785eb57 2024-09-21T14:21:23.532 INFO:teuthology.orchestra.run.smithi073.stdout:152 writesame 0x89c58f2 thru 0x89c9101 data_size 0x34(0x3810 bytes) 2024-09-21T14:21:23.533 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:23.534 INFO:teuthology.orchestra.run.smithi073.stdout:153 read 0xcfd2fe thru 0xd094c2 (0xc1c5 bytes) 2024-09-21T14:21:23.534 INFO:teuthology.orchestra.run.smithi073.stdout:156 writesame 0x5d80811 thru 0x5d85a22 data_size 0x37(0x5212 bytes) 2024-09-21T14:21:23.543 INFO:teuthology.orchestra.run.smithi073.stdout:157 write 0x382319a thru 0x3827aa6 (0x490d bytes) 2024-09-21T14:21:23.551 INFO:teuthology.orchestra.run.smithi073.stdout:160 read 0x4562eb7 thru 0x456d98a (0xaad4 bytes) 2024-09-21T14:21:23.553 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:23.553 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:23.564 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:23.564 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:23.564 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:23.564 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:23.565 INFO:teuthology.orchestra.run.smithi073.stdout:162 write 0xed53d1d thru 0xed60bcb (0xceaf bytes) 2024-09-21T14:21:23.569 INFO:teuthology.orchestra.run.smithi073.stdout:163 read 0xb869e85 thru 0xb86e7e8 (0x4964 bytes) 2024-09-21T14:21:23.569 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:23.570+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:23.571 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:23.575 INFO:teuthology.orchestra.run.smithi073.stdout:164 trunc from 0xed60bcc to 0xaa7720c 2024-09-21T14:21:23.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:23.589 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:23.589 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:23.594 INFO:teuthology.orchestra.run.smithi073.stdout:165 write 0xb30c6ae thru 0xb30d587 (0xeda bytes) 2024-09-21T14:21:23.599 INFO:teuthology.orchestra.run.smithi073.stdout:166 trunc from 0xb30d588 to 0xb67ee65 2024-09-21T14:21:23.604 INFO:teuthology.orchestra.run.smithi073.stdout:167 read 0x719de49 thru 0x71a1a8f (0x3c47 bytes) 2024-09-21T14:21:23.605 INFO:teuthology.orchestra.run.smithi073.stdout:168 writesame 0x6057393 thru 0x605ed35 data_size 0x1(0x79a3 bytes) 2024-09-21T14:21:23.607 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:23.607 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:23.618 INFO:teuthology.orchestra.run.smithi073.stdout:169 write 0x46e1dcc thru 0x46e2605 (0x83a bytes) 2024-09-21T14:21:23.625 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:23.625 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:23.634 INFO:teuthology.orchestra.run.smithi073.stdout:170 write 0xd7e54c0 thru 0xd7ee769 (0x92aa bytes) 2024-09-21T14:21:23.637 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xe77511b 2024-09-21T14:21:23.638 INFO:teuthology.orchestra.run.smithi073.stdout:171 trunc from 0xd7ee76a to 0xe77511b 2024-09-21T14:21:23.643 INFO:teuthology.orchestra.run.smithi073.stdout:174 write 0xbdad10 thru 0xbe4f8e (0xa27f bytes) 2024-09-21T14:21:23.644 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:23.644 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:23.660 INFO:teuthology.orchestra.run.smithi073.stdout:177 write 0x488a241 thru 0x48987c8 (0xe588 bytes) 2024-09-21T14:21:23.666 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:23.670 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:23.670+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:23.672 INFO:teuthology.orchestra.run.smithi073.stdout:178 clone 3 order 19 su 16384 sc 3 2024-09-21T14:21:23.702 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:23.702 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:23.702 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:23.703 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:23.703 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:23.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:23.795 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:23.795+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15)", 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.312516985, 2024-09-21T14:21:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015540700000000001, 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541788+0000", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541786+0000", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541792+0000", 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8830000000000003e-06 2024-09-21T14:21:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541800+0000", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1969999999999996e-06 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541807+0000", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9149999999999999e-06 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541854+0000", 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6987999999999998e-05 2024-09-21T14:21:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.541943+0000", 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8875000000000005e-05 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.318681696000001, 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027325829999999998, 2024-09-21T14:21:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 5 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535623+0000", 2024-09-21T14:21:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535621+0000", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535635+0000", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3899e-05 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535653+0000", 2024-09-21T14:21:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8423999999999998e-05 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535665+0000", 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1993e-05 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535851+0000", 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186119 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.535967+0000", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115701 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.536099+0000", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000132022 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.537334+0000", 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001234684 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538262+0000", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092820100000000003 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538316+0000", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4540000000000003e-05 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.538356+0000", 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9329999999999998e-05 2024-09-21T14:21:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4254.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.186010658000001, 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057806730000000001, 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4254", 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1337769221", 2024-09-21T14:21:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668294+0000", 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668290+0000", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999957 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668307+0000", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7039000000000001e-05 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668323+0000", 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6753e-05 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668336+0000", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2576e-05 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668402+0000", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5927999999999996e-05 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.668411+0000", 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3049999999999998e-06 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.672915+0000", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004503295 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.672978+0000", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3864999999999999e-05 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673080+0000", 2024-09-21T14:21:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101183 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673719+0000", 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063918899999999997 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673995+0000", 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00027601299999999997 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674038+0000", 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3214999999999999e-05 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674075+0000", 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6806e-05 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4254.0:7 1.2 e17/14 ondisk, result = 0)", 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.180381426, 2024-09-21T14:21:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000158071, 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673923+0000", 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673922+0000", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673927+0000", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7180000000000004e-06 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673937+0000", 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6870000000000006e-06 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673943+0000", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0719999999999996e-06 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673974+0000", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1239999999999999e-05 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.673980+0000", 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.657e-06 2024-09-21T14:21:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:10.674081+0000", 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100925 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:10 3.1c e22/19 3:3831950c:::rbd_directory:head v 22'1, mlcod=0'0)", 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.7050865370000001, 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093686500000000005, 2024-09-21T14:21:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149218+0000", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149223+0000", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149239+0000", 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6290000000000002e-05 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149250+0000", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0098000000000001e-05 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149312+0000", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2656000000000005e-05 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149333+0000", 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0996999999999999e-05 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150127+0000", 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000794255 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150155+0000", 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7654999999999999e-05 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:12 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6980499330000001, 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014151999999999999, 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 12 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156255+0000", 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156253+0000", 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156264+0000", 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0881e-05 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156281+0000", 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6929e-05 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156289+0000", 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2979999999999992e-06 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156346+0000", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6709999999999997e-05 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156504+0000", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000157999 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 1", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.156759+0000", 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025506200000000002 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157433+0000", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067386200000000001 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157573+0000", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014004100000000001 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157628+0000", 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5086000000000001e-05 2024-09-21T14:21:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157670+0000", 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2205e-05 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:12 3.f e22/19 ondisk, result = 0)", 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6968030220000001, 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017642800000000001, 2024-09-21T14:21:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157502+0000", 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157501+0000", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157506+0000", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7600000000000002e-06 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157515+0000", 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6020000000000006e-06 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157522+0000", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6980000000000001e-06 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157557+0000", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213000000000003e-05 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157566+0000", 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0319999999999992e-06 2024-09-21T14:21:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.157678+0000", 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000111877 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:16 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6857303890000002, 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032511399999999999, 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 16 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168574+0000", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168572+0000", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168585+0000", 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.322e-05 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168607+0000", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2042999999999999e-05 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168627+0000", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9241000000000001e-05 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168671+0000", 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4674e-05 2024-09-21T14:21:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168743+0000", 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1156999999999997e-05 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168899+0000", 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015698400000000001 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:17 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6853070369999998, 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017967100000000001, 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 17 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168998+0000", 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.168997+0000", 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169002+0000", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2349999999999999e-06 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169010+0000", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0509999999999997e-06 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169015+0000", 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169043+0000", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8243e-05 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169099+0000", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6326999999999998e-05 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169177+0000", 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8102999999999994e-05 2024-09-21T14:21:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:18 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6850454299999997, 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000174036, 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 18 2024-09-21T14:21:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169259+0000", 2024-09-21T14:21:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169263+0000", 2024-09-21T14:21:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:21:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169271+0000", 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3250000000000008e-06 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169276+0000", 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1640000000000003e-06 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169298+0000", 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2829000000000001e-05 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169342+0000", 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3488000000000001e-05 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169433+0000", 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1500000000000001e-05 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:19 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6847700969999999, 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000146105, 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 19 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169535+0000", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169534+0000", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169539+0000", 2024-09-21T14:21:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9099999999999996e-06 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169549+0000", 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0597999999999999e-05 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169556+0000", 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.046e-06 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169581+0000", 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4454e-05 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169620+0000", 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9397999999999998e-05 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.169681+0000", 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0547999999999997e-05 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:20 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6842095669999999, 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031777800000000001, 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 20 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170095+0000", 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170104+0000", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5079999999999994e-06 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170113+0000", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0480000000000004e-06 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170119+0000", 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3550000000000001e-06 2024-09-21T14:21:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170152+0000", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2268999999999999e-05 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170202+0000", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0639000000000001e-05 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170413+0000", 2024-09-21T14:21:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021063200000000001 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.683783509, 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291587, 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170521+0000", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170525+0000", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3120000000000004e-06 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170533+0000", 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0369999999999995e-06 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170539+0000", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170564+0000", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5480999999999999e-05 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170625+0000", 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1007000000000001e-05 2024-09-21T14:21:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170813+0000", 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018774899999999999 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:22 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6833980090000003, 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023776599999999999, 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 22 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170907+0000", 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170906+0000", 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170910+0000", 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1350000000000002e-06 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170921+0000", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0974e-05 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170927+0000", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7939999999999999e-06 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.170956+0000", 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8755000000000002e-05 2024-09-21T14:21:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171028+0000", 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2043000000000001e-05 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171145+0000", 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011680600000000001 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:24 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6826843199999999, 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019065999999999999, 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 24 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171620+0000", 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171625+0000", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6400000000000002e-06 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171636+0000", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0628999999999999e-05 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171650+0000", 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3964e-05 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171682+0000", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2014999999999998e-05 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171732+0000", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0204999999999998e-05 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171811+0000", 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9083000000000002e-05 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:25 3.f 3:f0449c44:::rbd_header.1121746b0c91:head [watch watch cookie 140451940236640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6823378710000001, 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0009969, 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 25 2024-09-21T14:21:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171967+0000", 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171966+0000", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171971+0000", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9559999999999996e-06 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.171981+0000", 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0182e-05 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172029+0000", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7817e-05 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172060+0000", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0568000000000003e-05 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172104+0000", 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4295000000000002e-05 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 1", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172198+0000", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4034000000000001e-05 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172715+0000", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051702900000000001 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172900+0000", 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018541599999999999 2024-09-21T14:21:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172927+0000", 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6409000000000001e-05 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172964+0000", 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6971999999999998e-05 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:25 3.f e22/19 ondisk, result = 0)", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6814702879999999, 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013298800000000001, 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172834+0000", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172839+0000", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1000000000000003e-06 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172850+0000", 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1260999999999999e-05 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172864+0000", 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4121000000000001e-05 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172890+0000", 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5783999999999999e-05 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172897+0000", 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2470000000000002e-06 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.172967+0000", 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0358000000000006e-05 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:1391 3.13 e25/19 3:cd6406cc:::rbd_data.11212f271376.0000000000000008:head v 25'4, mlcod=25'2)", 2024-09-21T14:21:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.551114+0000", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.303190925, 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00089582099999999999, 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551114+0000", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551114+0000", 2024-09-21T14:21:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551113+0000", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551150+0000", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7202000000000001e-05 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551167+0000", 2024-09-21T14:21:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7204e-05 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551177+0000", 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9990000000000003e-06 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551229+0000", 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1756999999999999e-05 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551254+0000", 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5534e-05 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.551971+0000", 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00071629400000000002 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.552010+0000", 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9066999999999997e-05 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1550 3.15 3:ab811006:::rbd_data.11212f271376.000000000000001d:head [set-alloc-hint object_size 8388608 write_size 8388608,write 3298496~37546] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.636833+0000", 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.21747156400000001, 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015038689999999999, 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1550 2024-09-21T14:21:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636833+0000", 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636833+0000", 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636832+0000", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636867+0000", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5284000000000003e-05 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636882+0000", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5583999999999999e-05 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636890+0000", 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2350000000000002e-06 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.636936+0000", 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5933000000000002e-05 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.637030+0000", 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3956999999999995e-05 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.637115+0000", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.637821+0000", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00070676599999999995 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638243+0000", 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000421758 2024-09-21T14:21:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638296+0000", 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.321e-05 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638337+0000", 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0776999999999999e-05 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop_reply(client.4385.0:1550 3.15 e25/19 ondisk, result = 0)", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.638173+0000", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.216132138, 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000173055, 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638173+0000", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638173+0000", 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638172+0000", 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638177+0000", 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6230000000000002e-06 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638186+0000", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4190000000000002e-06 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638190+0000", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2919999999999997e-06 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638226+0000", 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5358999999999998e-05 2024-09-21T14:21:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638235+0000", 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2599999999999994e-06 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.638346+0000", 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110865 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:23.948 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:23.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:23.950 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:23.949+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:24.051 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:24.051+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:24.092 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.092 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:24.092 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:24.092 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.100 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:24.152 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:24.152+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:21:24.181 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:21:24.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:24.210 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:24.210 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:24.225 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone2 from 0xe77511b (overlap 0x785eb57) to 0x0 2024-09-21T14:21:24.253 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:24.252+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:24.348 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.348 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4254.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.810804665999999, 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018998559999999999, 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536822+0000", 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536834+0000", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2065000000000001e-05 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536845+0000", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1073000000000001e-05 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536853+0000", 2024-09-21T14:21:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8029999999999997e-06 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536895+0000", 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2753999999999998e-05 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.536917+0000", 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.192e-05 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.538696+0000", 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001778473 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:08.538722+0000", 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6336999999999999e-05 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.2045874190000001, 2024-09-21T14:21:24.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029151810000000002, 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143037+0000", 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143052+0000", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.449e-05 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143070+0000", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8105e-05 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143082+0000", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2045000000000001e-05 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143128+0000", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6109999999999997e-05 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143133+0000", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5449999999999997e-06 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144568+0000", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001435497 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144692+0000", 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012378700000000001 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144830+0000", 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138042 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145538+0000", 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00070826400000000001 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145872+0000", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033409800000000001 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145918+0000", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5388e-05 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145955+0000", 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6874e-05 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.359 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.2031171839999999, 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039806640000000001, 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.360 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144507+0000", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144519+0000", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1284000000000001e-05 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144535+0000", 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144545+0000", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9110000000000005e-06 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144595+0000", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0318999999999997e-05 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144697+0000", 2024-09-21T14:21:24.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010137999999999999 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144844+0000", 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014769800000000001 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.147490+0000", 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026452289999999998 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148368+0000", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000878738 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148425+0000", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7167999999999999e-05 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148490+0000", 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4828999999999999e-05 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.365 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.1989918079999997, 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048317120000000002, 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10 2024-09-21T14:21:24.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148634+0000", 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148643+0000", 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8289999999999997e-06 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148652+0000", 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.312e-06 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148659+0000", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9800000000000001e-06 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148708+0000", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9147999999999998e-05 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148852+0000", 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014422 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149086+0000", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023317999999999999 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150494+0000", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001408402 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153356+0000", 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028617360000000001 2024-09-21T14:21:24.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153421+0000", 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5625999999999998e-05 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153467+0000", 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5402999999999999e-05 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:11 3.1f e22/19 3:fa654f18:::rbd_id.image_client.0:head v 22'1, mlcod=0'0)", 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.1936159479999997, 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018756840000000001, 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154011+0000", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154010+0000", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154019+0000", 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7590000000000003e-06 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154029+0000", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0546e-05 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154045+0000", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5739e-05 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154081+0000", 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5806000000000002e-05 2024-09-21T14:21:24.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.154098+0000", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7717e-05 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.155838+0000", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001739263 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.155887+0000", 2024-09-21T14:21:24.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8807000000000002e-05 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1402 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.566307+0000", 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.78131995200000004, 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019937560000000002, 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.376 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1402 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566307+0000", 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566307+0000", 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566306+0000", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566311+0000", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566320+0000", 2024-09-21T14:21:24.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0589999999999992e-06 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566324+0000", 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4510000000000002e-06 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566362+0000", 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7696e-05 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566448+0000", 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6003999999999999e-05 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566828+0000", 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038024000000000001 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568158+0000", 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013298030000000001 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568205+0000", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7608999999999998e-05 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568258+0000", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2451999999999999e-05 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568301+0000", 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2701999999999999e-05 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.381 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1403 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.566338+0000", 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.78128871300000002, 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0023336580000000002, 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1403 2024-09-21T14:21:24.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566338+0000", 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566338+0000", 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566338+0000", 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566340+0000", 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.272e-06 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566344+0000", 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.216e-06 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.566348+0000", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.771e-06 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.567195+0000", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00084693099999999999 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.567294+0000", 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8697999999999996e-05 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.567752+0000", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00045843899999999999 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568481+0000", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072926700000000002 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568623+0000", 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014174000000000001 2024-09-21T14:21:24.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568651+0000", 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8031999999999999e-05 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.568672+0000", 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0652000000000001e-05 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:1410 3.19 e25/19 3:9df50081:::rbd_header.11212f271376:head v 25'21, mlcod=25'19)", 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.581188+0000", 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.766438497, 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022339819999999998, 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581188+0000", 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581188+0000", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581188+0000", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581201+0000", 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3097e-05 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581209+0000", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8699999999999992e-06 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581213+0000", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5129999999999998e-06 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581256+0000", 2024-09-21T14:21:24.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2354999999999997e-05 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.581279+0000", 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3753000000000001e-05 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583393+0000", 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002113939 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583422+0000", 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9014999999999999e-05 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1411 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.76394199299999999, 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026751169999999999, 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1411 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583684+0000", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583688+0000", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7239999999999998e-06 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583696+0000", 2024-09-21T14:21:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1440000000000003e-06 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583701+0000", 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.527e-06 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583733+0000", 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2721000000000001e-05 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583786+0000", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2355000000000003e-05 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583940+0000", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015385 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.584953+0000", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010138580000000001 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586305+0000", 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013520089999999999 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586336+0000", 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0846999999999998e-05 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586360+0000", 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3689999999999998e-05 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:1420 3.19 e25/19 3:9df50081:::rbd_header.11212f271376:head v 25'22, mlcod=25'20)", 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.590981+0000", 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.75664600500000001, 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019742990000000001, 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.590981+0000", 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.590981+0000", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.590980+0000", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.590993+0000", 2024-09-21T14:21:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2432999999999999e-05 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.591002+0000", 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0329999999999997e-06 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.591006+0000", 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2899999999999996e-06 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.591050+0000", 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3614e-05 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.591070+0000", 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0735000000000001e-05 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.592914+0000", 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001843765 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.592955+0000", 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1087999999999997e-05 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:1429 3.d e25/19 3:b70b73b0:::rbd_data.11212f271376.0000000000000018:head v 25'1, mlcod=0'0)", 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.597667+0000", 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.74996005399999999, 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002157235, 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597667+0000", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597667+0000", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597666+0000", 2024-09-21T14:21:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597677+0000", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0737000000000001e-05 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597689+0000", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1864e-05 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597695+0000", 2024-09-21T14:21:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4300000000000003e-06 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597742+0000", 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6952e-05 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.597762+0000", 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0502000000000001e-05 2024-09-21T14:21:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.599785+0000", 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002022477 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.599824+0000", 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9008999999999999e-05 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1430 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.600450+0000", 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.74717645499999996, 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021051249999999998, 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1430 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600450+0000", 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600450+0000", 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600449+0000", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600457+0000", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.422e-06 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600470+0000", 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2599000000000001e-05 2024-09-21T14:21:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600483+0000", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3399e-05 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600524+0000", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0809000000000002e-05 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600593+0000", 2024-09-21T14:21:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9039000000000001e-05 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.600772+0000", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017878300000000001 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.601543+0000", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077129300000000004 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.602458+0000", 2024-09-21T14:21:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091527999999999998 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.602515+0000", 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6221999999999998e-05 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.602555+0000", 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0936999999999998e-05 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1739 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.649951+0000", 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.69767576799999997, 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018567869999999999, 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1739 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649951+0000", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649951+0000", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649949+0000", 2024-09-21T14:21:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649958+0000", 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7520000000000002e-06 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649976+0000", 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8348000000000001e-05 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.649991+0000", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4748e-05 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.650048+0000", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.66e-05 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.650153+0000", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105196 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.650575+0000", 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00042158799999999998 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.651430+0000", 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00085583399999999998 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.651724+0000", 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000293484 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.651773+0000", 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9357000000000002e-05 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.651808+0000", 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4573000000000003e-05 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1741 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.69405051399999995, 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030142189999999998, 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1741 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653575+0000", 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653581+0000", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3399999999999997e-06 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653591+0000", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0144e-05 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653596+0000", 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9470000000000002e-06 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653627+0000", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1130999999999997e-05 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653678+0000", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1227000000000003e-05 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653887+0000", 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020935800000000001 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.654611+0000", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072371200000000005 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656495+0000", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018841280000000001 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656550+0000", 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4264000000000001e-05 2024-09-21T14:21:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656591+0000", 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0929000000000001e-05 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1742 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2469136~21232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.69097583600000001, 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041457020000000002, 2024-09-21T14:21:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1742 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656650+0000", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656674+0000", 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4219999999999999e-05 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656683+0000", 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7479999999999999e-06 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656688+0000", 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7119999999999999e-06 2024-09-21T14:21:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656733+0000", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4846999999999999e-05 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656871+0000", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137933 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.657079+0000", 2024-09-21T14:21:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020842699999999999 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.657873+0000", 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079421600000000004 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660706+0000", 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028326319999999999 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660759+0000", 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3529999999999997e-05 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660797+0000", 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7178000000000002e-05 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1935 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.666416+0000", 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.68121126600000004, 2024-09-21T14:21:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001952021, 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1935 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666416+0000", 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666416+0000", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666414+0000", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666425+0000", 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1318000000000001e-05 2024-09-21T14:21:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666443+0000", 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7815000000000001e-05 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666452+0000", 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8199999999999992e-06 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666506+0000", 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3943999999999997e-05 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666600+0000", 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3363999999999996e-05 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.666999+0000", 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039895499999999999 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.667804+0000", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00080568899999999995 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.668272+0000", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000467381 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.668324+0000", 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2417999999999999e-05 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.668368+0000", 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3321000000000001e-05 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1937 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.670197+0000", 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.67743028400000005, 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015784130000000001, 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1937 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670197+0000", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670197+0000", 2024-09-21T14:21:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670195+0000", 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670205+0000", 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.057e-05 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670221+0000", 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6008e-05 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670229+0000", 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4900000000000003e-06 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670286+0000", 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7024000000000002e-05 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670396+0000", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110287 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.670835+0000", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00043936399999999998 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.671669+0000", 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083370500000000001 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.671678+0000", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8939999999999999e-06 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.671728+0000", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9904999999999997e-05 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.671775+0000", 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7126000000000002e-05 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2030 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.119502302, 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052215689999999997, 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2030 2024-09-21T14:21:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228123+0000", 2024-09-21T14:21:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228129+0000", 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1190000000000002e-06 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228144+0000", 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4198e-05 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228151+0000", 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4789999999999999e-06 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228209+0000", 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8032999999999998e-05 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228300+0000", 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0861999999999998e-05 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228494+0000", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019448999999999999 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.231566+0000", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003072052 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233247+0000", 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016804929999999999 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233303+0000", 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6005999999999999e-05 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233346+0000", 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3072999999999999e-05 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.a epoch 26/19 rep_tid 229 entries 26'48 (0'0) error 3:552b5ba1:::rbd_data.11212f271376.000000000000000e:head by client.4385.0:2501 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'46)", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238069+0000", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.10955770200000001, 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002652101, 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238069+0000", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238069+0000", 2024-09-21T14:21:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238068+0000", 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238080+0000", 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1656e-05 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238102+0000", 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1882000000000002e-05 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238115+0000", 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2917e-05 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238199+0000", 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4908999999999996e-05 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240721+0000", 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025217619999999999 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2598 3.1a 3:5e76b464:::rbd_data.11212f271376.000000000000000f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.239473+0000", 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.10815330200000001, 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002314628, 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2598 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239473+0000", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239473+0000", 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239472+0000", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239480+0000", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9629999999999998e-06 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239495+0000", 2024-09-21T14:21:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5102999999999999e-05 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239502+0000", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4460000000000004e-06 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239550+0000", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7889000000000002e-05 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239635+0000", 2024-09-21T14:21:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4349000000000002e-05 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.239751+0000", 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011649700000000001 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.241243+0000", 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001492207 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.241700+0000", 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00045709400000000001 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.241745+0000", 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4968e-05 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.241788+0000", 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2793000000000002e-05 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.458 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:24.459 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:24.459+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:24.505 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:24.509 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T14:21:24.519 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T14:21:24.421166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95445985,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":978,"num_read_kb":1172,"num_write":314,"num_write_kb":3395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218220,"kb_used_data":2040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756500,"statfs":{"total":773094113280,"available":772870656000,"internally_reserved":0,"allocated":2088960,"data_stored":1925383,"data_compressed":66423,"data_compressed_allocated":94208,"data_compressed_original":1179648,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000562"},"pg_stats":[{"pgid":"3.1f","version":"22'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034921+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:22.034921+0000","last_peered":"2024-09-21T14:21:22.034921+0000","last_clean":"2024-09-21T14:21:22.034921+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:22.034921+0000","last_undegraded":"2024-09-21T14:21:22.034921+0000","last_fullsized":"2024-09-21T14:21:22.034921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"23'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.294461+0000","last_change":"2024-09-21T14:21:20.149718+0000","last_active":"2024-09-21T14:21:22.294461+0000","last_peered":"2024-09-21T14:21:22.294461+0000","last_clean":"2024-09-21T14:21:22.294461+0000","last_became_active":"2024-09-21T14:21:18.152993+0000","last_became_peered":"2024-09-21T14:21:18.152993+0000","last_unstale":"2024-09-21T14:21:22.294461+0000","last_undegraded":"2024-09-21T14:21:22.294461+0000","last_fullsized":"2024-09-21T14:21:22.294461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:18.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":3207094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":31,"num_write":1,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152252+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:21.152252+0000","last_peered":"2024-09-21T14:21:21.152252+0000","last_clean":"2024-09-21T14:21:21.152252+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:21.152252+0000","last_undegraded":"2024-09-21T14:21:21.152252+0000","last_fullsized":"2024-09-21T14:21:21.152252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"24'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.313966+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:22.313966+0000","last_peered":"2024-09-21T14:21:22.313966+0000","last_clean":"2024-09-21T14:21:22.313966+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:22.313966+0000","last_undegraded":"2024-09-21T14:21:22.313966+0000","last_fullsized":"2024-09-21T14:21:22.313966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":3995317,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":18,"num_write":3,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975561+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:21.975561+0000","last_peered":"2024-09-21T14:21:21.975561+0000","last_clean":"2024-09-21T14:21:21.975561+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:21.975561+0000","last_undegraded":"2024-09-21T14:21:21.975561+0000","last_fullsized":"2024-09-21T14:21:21.975561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"22'2","reported_seq":47,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.382719+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:22.382719+0000","last_peered":"2024-09-21T14:21:22.382719+0000","last_clean":"2024-09-21T14:21:22.382719+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:22.382719+0000","last_undegraded":"2024-09-21T14:21:22.382719+0000","last_fullsized":"2024-09-21T14:21:22.382719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"24'4","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418472+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:22.418472+0000","last_peered":"2024-09-21T14:21:22.418472+0000","last_clean":"2024-09-21T14:21:22.418472+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:22.418472+0000","last_undegraded":"2024-09-21T14:21:22.418472+0000","last_fullsized":"2024-09-21T14:21:22.418472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":7534650,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":1,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.409252+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:22.409252+0000","last_peered":"2024-09-21T14:21:22.409252+0000","last_clean":"2024-09-21T14:21:22.409252+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:22.409252+0000","last_undegraded":"2024-09-21T14:21:22.409252+0000","last_fullsized":"2024-09-21T14:21:22.409252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":13184888,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":1,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034956+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:22.034956+0000","last_peered":"2024-09-21T14:21:22.034956+0000","last_clean":"2024-09-21T14:21:22.034956+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:22.034956+0000","last_undegraded":"2024-09-21T14:21:22.034956+0000","last_fullsized":"2024-09-21T14:21:22.034956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151529+0000","last_change":"2024-09-21T14:21:20.755882+0000","last_active":"2024-09-21T14:21:21.151529+0000","last_peered":"2024-09-21T14:21:21.151529+0000","last_clean":"2024-09-21T14:21:21.151529+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.151529+0000","last_undegraded":"2024-09-21T14:21:21.151529+0000","last_fullsized":"2024-09-21T14:21:21.151529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:17.531857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.256146+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:21.256146+0000","last_peered":"2024-09-21T14:21:21.256146+0000","last_clean":"2024-09-21T14:21:21.256146+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:21.256146+0000","last_undegraded":"2024-09-21T14:21:21.256146+0000","last_fullsized":"2024-09-21T14:21:21.256146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152175+0000","last_change":"2024-09-21T14:21:20.161641+0000","last_active":"2024-09-21T14:21:21.152175+0000","last_peered":"2024-09-21T14:21:21.152175+0000","last_clean":"2024-09-21T14:21:21.152175+0000","last_became_active":"2024-09-21T14:21:18.152393+0000","last_became_peered":"2024-09-21T14:21:18.152393+0000","last_unstale":"2024-09-21T14:21:21.152175+0000","last_undegraded":"2024-09-21T14:21:21.152175+0000","last_fullsized":"2024-09-21T14:21:21.152175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:55.413999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.877129+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:21.877129+0000","last_peered":"2024-09-21T14:21:21.877129+0000","last_clean":"2024-09-21T14:21:21.877129+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:21.877129+0000","last_undegraded":"2024-09-21T14:21:21.877129+0000","last_fullsized":"2024-09-21T14:21:21.877129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'16","reported_seq":96,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.424523+0000","last_change":"2024-09-21T14:21:20.755817+0000","last_active":"2024-09-21T14:21:22.424523+0000","last_peered":"2024-09-21T14:21:22.424523+0000","last_clean":"2024-09-21T14:21:22.424523+0000","last_became_active":"2024-09-21T14:21:18.150612+0000","last_became_peered":"2024-09-21T14:21:18.150612+0000","last_unstale":"2024-09-21T14:21:22.424523+0000","last_undegraded":"2024-09-21T14:21:22.424523+0000","last_fullsized":"2024-09-21T14:21:22.424523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:50.934485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":180,"num_read_kb":154,"num_write":15,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152523+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:21.152523+0000","last_peered":"2024-09-21T14:21:21.152523+0000","last_clean":"2024-09-21T14:21:21.152523+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:21.152523+0000","last_undegraded":"2024-09-21T14:21:21.152523+0000","last_fullsized":"2024-09-21T14:21:21.152523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.927326+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:21.927326+0000","last_peered":"2024-09-21T14:21:21.927326+0000","last_clean":"2024-09-21T14:21:21.927326+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:21.927326+0000","last_undegraded":"2024-09-21T14:21:21.927326+0000","last_fullsized":"2024-09-21T14:21:21.927326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151554+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:21.151554+0000","last_peered":"2024-09-21T14:21:21.151554+0000","last_clean":"2024-09-21T14:21:21.151554+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:21.151554+0000","last_undegraded":"2024-09-21T14:21:21.151554+0000","last_fullsized":"2024-09-21T14:21:21.151554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875923+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:21.875923+0000","last_peered":"2024-09-21T14:21:21.875923+0000","last_clean":"2024-09-21T14:21:21.875923+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:21.875923+0000","last_undegraded":"2024-09-21T14:21:21.875923+0000","last_fullsized":"2024-09-21T14:21:21.875923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.947981+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:21.947981+0000","last_peered":"2024-09-21T14:21:21.947981+0000","last_clean":"2024-09-21T14:21:21.947981+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:21.947981+0000","last_undegraded":"2024-09-21T14:21:21.947981+0000","last_fullsized":"2024-09-21T14:21:21.947981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'6","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.416170+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:22.416170+0000","last_peered":"2024-09-21T14:21:22.416170+0000","last_clean":"2024-09-21T14:21:22.416170+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:22.416170+0000","last_undegraded":"2024-09-21T14:21:22.416170+0000","last_fullsized":"2024-09-21T14:21:22.416170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":15209098,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":125,"num_write":3,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975446+0000","last_change":"2024-09-21T14:21:20.755906+0000","last_active":"2024-09-21T14:21:21.975446+0000","last_peered":"2024-09-21T14:21:21.975446+0000","last_clean":"2024-09-21T14:21:21.975446+0000","last_became_active":"2024-09-21T14:21:18.151157+0000","last_became_peered":"2024-09-21T14:21:18.151157+0000","last_unstale":"2024-09-21T14:21:21.975446+0000","last_undegraded":"2024-09-21T14:21:21.975446+0000","last_fullsized":"2024-09-21T14:21:21.975446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:25.967224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152075+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:21.152075+0000","last_peered":"2024-09-21T14:21:21.152075+0000","last_clean":"2024-09-21T14:21:21.152075+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:21.152075+0000","last_undegraded":"2024-09-21T14:21:21.152075+0000","last_fullsized":"2024-09-21T14:21:21.152075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875956+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:21.875956+0000","last_peered":"2024-09-21T14:21:21.875956+0000","last_clean":"2024-09-21T14:21:21.875956+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:21.875956+0000","last_undegraded":"2024-09-21T14:21:21.875956+0000","last_fullsized":"2024-09-21T14:21:21.875956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034849+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:22.034849+0000","last_peered":"2024-09-21T14:21:22.034849+0000","last_clean":"2024-09-21T14:21:22.034849+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:22.034849+0000","last_undegraded":"2024-09-21T14:21:22.034849+0000","last_fullsized":"2024-09-21T14:21:22.034849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.255796+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:21.255796+0000","last_peered":"2024-09-21T14:21:21.255796+0000","last_clean":"2024-09-21T14:21:21.255796+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:21.255796+0000","last_undegraded":"2024-09-21T14:21:21.255796+0000","last_fullsized":"2024-09-21T14:21:21.255796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"24'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.182951+0000","last_change":"2024-09-21T14:21:20.151933+0000","last_active":"2024-09-21T14:21:22.182951+0000","last_peered":"2024-09-21T14:21:22.182951+0000","last_clean":"2024-09-21T14:21:22.182951+0000","last_became_active":"2024-09-21T14:21:18.147902+0000","last_became_peered":"2024-09-21T14:21:18.147902+0000","last_unstale":"2024-09-21T14:21:22.182951+0000","last_undegraded":"2024-09-21T14:21:22.182951+0000","last_fullsized":"2024-09-21T14:21:22.182951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:09.190450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":41,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152478+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:21.152478+0000","last_peered":"2024-09-21T14:21:21.152478+0000","last_clean":"2024-09-21T14:21:21.152478+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:21.152478+0000","last_undegraded":"2024-09-21T14:21:21.152478+0000","last_fullsized":"2024-09-21T14:21:21.152478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"24'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.355019+0000","last_change":"2024-09-21T14:21:20.145954+0000","last_active":"2024-09-21T14:21:22.355019+0000","last_peered":"2024-09-21T14:21:22.355019+0000","last_clean":"2024-09-21T14:21:22.355019+0000","last_became_active":"2024-09-21T14:21:18.152479+0000","last_became_peered":"2024-09-21T14:21:18.152479+0000","last_unstale":"2024-09-21T14:21:22.355019+0000","last_undegraded":"2024-09-21T14:21:22.355019+0000","last_fullsized":"2024-09-21T14:21:22.355019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:27.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":16777232,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":6,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875848+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:21.875848+0000","last_peered":"2024-09-21T14:21:21.875848+0000","last_clean":"2024-09-21T14:21:21.875848+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:21.875848+0000","last_undegraded":"2024-09-21T14:21:21.875848+0000","last_fullsized":"2024-09-21T14:21:21.875848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"24'37","reported_seq":59,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418646+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:22.418646+0000","last_peered":"2024-09-21T14:21:22.418646+0000","last_clean":"2024-09-21T14:21:22.418646+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:22.418646+0000","last_undegraded":"2024-09-21T14:21:22.418646+0000","last_fullsized":"2024-09-21T14:21:22.418646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":6837908,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":194,"num_write":53,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"24'17","reported_seq":35,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.379636+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:22.379636+0000","last_peered":"2024-09-21T14:21:22.379636+0000","last_clean":"2024-09-21T14:21:22.379636+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:22.379636+0000","last_undegraded":"2024-09-21T14:21:22.379636+0000","last_fullsized":"2024-09-21T14:21:22.379636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":36,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":51,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"24'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.354567+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:22.354567+0000","last_peered":"2024-09-21T14:21:22.354567+0000","last_clean":"2024-09-21T14:21:22.354567+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:22.354567+0000","last_undegraded":"2024-09-21T14:21:22.354567+0000","last_fullsized":"2024-09-21T14:21:22.354567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":16167918,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":35,"num_write":3,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"23'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.345351+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:22.345351+0000","last_peered":"2024-09-21T14:21:22.345351+0000","last_clean":"2024-09-21T14:21:22.345351+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:22.345351+0000","last_undegraded":"2024-09-21T14:21:22.345351+0000","last_fullsized":"2024-09-21T14:21:22.345351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":1593100,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152309+0000","last_change":"2024-09-21T14:21:20.162015+0000","last_active":"2024-09-21T14:21:21.152309+0000","last_peered":"2024-09-21T14:21:21.152309+0000","last_clean":"2024-09-21T14:21:21.152309+0000","last_became_active":"2024-09-21T14:21:18.143760+0000","last_became_peered":"2024-09-21T14:21:18.143760+0000","last_unstale":"2024-09-21T14:21:21.152309+0000","last_undegraded":"2024-09-21T14:21:21.152309+0000","last_fullsized":"2024-09-21T14:21:21.152309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"23'4","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.015396+0000","last_change":"2024-09-21T14:21:20.756021+0000","last_active":"2024-09-21T14:21:22.015396+0000","last_peered":"2024-09-21T14:21:22.015396+0000","last_clean":"2024-09-21T14:21:22.015396+0000","last_became_active":"2024-09-21T14:21:18.152859+0000","last_became_peered":"2024-09-21T14:21:18.152859+0000","last_unstale":"2024-09-21T14:21:22.015396+0000","last_undegraded":"2024-09-21T14:21:22.015396+0000","last_fullsized":"2024-09-21T14:21:22.015396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.438917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"24'21","reported_seq":163,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.285127+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:22.285127+0000","last_peered":"2024-09-21T14:21:22.285127+0000","last_clean":"2024-09-21T14:21:22.285127+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:22.285127+0000","last_undegraded":"2024-09-21T14:21:22.285127+0000","last_fullsized":"2024-09-21T14:21:22.285127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":359,"num_read_kb":311,"num_write":19,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.426849+0000","last_change":"2024-09-21T14:21:20.160536+0000","last_active":"2024-09-21T14:21:22.426849+0000","last_peered":"2024-09-21T14:21:22.426849+0000","last_clean":"2024-09-21T14:21:22.426849+0000","last_became_active":"2024-09-21T14:21:18.151161+0000","last_became_peered":"2024-09-21T14:21:18.151161+0000","last_unstale":"2024-09-21T14:21:22.426849+0000","last_undegraded":"2024-09-21T14:21:22.426849+0000","last_fullsized":"2024-09-21T14:21:22.426849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:48.604891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":10348281,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":119,"num_write":8,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.958641+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:21.958641+0000","last_peered":"2024-09-21T14:21:21.958641+0000","last_clean":"2024-09-21T14:21:21.958641+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.958641+0000","last_undegraded":"2024-09-21T14:21:21.958641+0000","last_fullsized":"2024-09-21T14:21:21.958641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152151+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:21.152151+0000","last_peered":"2024-09-21T14:21:21.152151+0000","last_clean":"2024-09-21T14:21:21.152151+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:21.152151+0000","last_undegraded":"2024-09-21T14:21:21.152151+0000","last_fullsized":"2024-09-21T14:21:21.152151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":527422,"data_compressed":18878,"data_compressed_allocated":49152,"data_compressed_original":385024,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":83424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":92810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":672841,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":83473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":679631,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":89233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":134719,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":89252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":61208,"data_compressed":2228,"data_compressed_allocated":4096,"data_compressed_original":45056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":145636,"data_compressed":6414,"data_compressed_allocated":16384,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":66052,"data_compressed":3021,"data_compressed_allocated":8192,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":162317,"data_compressed":6046,"data_compressed_allocated":16384,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":31721,"data_compressed":1169,"data_compressed_allocated":4096,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":10839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:24.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:24.559 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:24.558+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:24.616 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.616 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:24.616 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:24.616 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:24.670 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:24.670+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:24.735 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.735 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:24.735 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:24.735 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:24.735 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.743 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:24.746 INFO:teuthology.orchestra.run.smithi073.stdout:179 read 0x8a36752 thru 0x8a449ab (0xe25a bytes) 2024-09-21T14:21:24.748 INFO:teuthology.orchestra.run.smithi073.stdout:181 punch from 0x7b1ba94 to 0x7b2af07, (0xf473 bytes) 2024-09-21T14:21:24.751 INFO:teuthology.orchestra.run.smithi073.stdout:182 write 0x1c5a0e4 thru 0x1c68f7d (0xee9a bytes) 2024-09-21T14:21:24.760 INFO:teuthology.orchestra.run.smithi073.stdout:183 write 0x2c7130c thru 0x2c765a9 (0x529e bytes) 2024-09-21T14:21:24.761 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:24.760+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:24.775 INFO:teuthology.orchestra.run.smithi073.stdout:185 trunc from 0xe77511b to 0x7438cff 2024-09-21T14:21:24.806 INFO:teuthology.orchestra.run.smithi073.stdout:186 write 0xc1569d1 thru 0xc15cfbd (0x65ed bytes) 2024-09-21T14:21:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:187 writesame 0xd31cb7b thru 0xd32c3b7 data_size 0x45(0xf83d bytes) 2024-09-21T14:21:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:189 writesame 0x7043442 thru 0x7047314 data_size 0x3(0x3ed3 bytes) 2024-09-21T14:21:24.862 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:24.862+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1975 3.5 3:a0d5fcf9:::rbd_header.11219861e469:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.192182+0000", 2024-09-21T14:21:24.869 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.67593302399999999, 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001963869, 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1975 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192182+0000", 2024-09-21T14:21:24.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192182+0000", 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192180+0000", 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192191+0000", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1311999999999999e-05 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192208+0000", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6342999999999999e-05 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192217+0000", 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7659999999999993e-06 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192262+0000", 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4944000000000002e-05 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.192268+0000", 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3109999999999996e-06 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.193944+0000", 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016768200000000001 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.194027+0000", 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2707999999999995e-05 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.194146+0000", 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000118411 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2027 3.5 3:a36395c2:::rbd_data.11212f271376.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.228120+0000", 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.63999430999999996, 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025991009999999999, 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2027 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228120+0000", 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228120+0000", 2024-09-21T14:21:24.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228118+0000", 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228128+0000", 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4490000000000005e-06 2024-09-21T14:21:24.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228148+0000", 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0616999999999999e-05 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228158+0000", 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9939999999999995e-06 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228220+0000", 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1940000000000007e-05 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228359+0000", 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138555 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230720+0000", 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0023608380000000001 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.879 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.2 epoch 26/19 rep_tid 62 entries 26'15 (0'0) error 3:4123907e:::rbd_data.11212f271376.0000000000000002:head by client.4385.0:2028 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'13)", 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.229456+0000", 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.63865858099999995, 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0038377760000000002, 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229456+0000", 2024-09-21T14:21:24.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229456+0000", 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229455+0000", 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229465+0000", 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1889999999999996e-06 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229485+0000", 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0287000000000001e-05 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229496+0000", 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1219e-05 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229560+0000", 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4372e-05 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233294+0000", 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037334989999999999 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.1 epoch 26/19 rep_tid 7 entries 26'3 (0'0) error 3:83f80fa9:::rbd_data.11212f271376.0000000000000003:head by client.4385.0:2029 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 23'1)", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.229644+0000", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.63847093799999999, 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036296649999999998, 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229644+0000", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229644+0000", 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229643+0000", 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229651+0000", 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3459999999999996e-06 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229663+0000", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.238e-05 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229673+0000", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7189999999999996e-06 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.229724+0000", 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1758999999999997e-05 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233273+0000", 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003549038 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:2035 3.2 e26/19 3:461cc5e4:::rbd_data.11212f271376.0000000000000009:head v 26'17, mlcod=24'13)", 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.230276+0000", 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.63783876699999997, 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033974140000000001, 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230276+0000", 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230276+0000", 2024-09-21T14:21:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230275+0000", 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230284+0000", 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1050000000000001e-06 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230297+0000", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2214e-05 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230303+0000", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4760000000000003e-06 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230337+0000", 2024-09-21T14:21:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4434000000000003e-05 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.230368+0000", 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0895000000000002e-05 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233640+0000", 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003271553 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233673+0000", 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3476999999999998e-05 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2599 3.5 3:a36395c2:::rbd_data.11212f271376.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.240084+0000", 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.62803097200000002, 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024143120000000001, 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2599 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240084+0000", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240084+0000", 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240081+0000", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240100+0000", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8502e-05 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240128+0000", 2024-09-21T14:21:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7943e-05 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240138+0000", 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8430000000000005e-06 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240187+0000", 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9357999999999997e-05 2024-09-21T14:21:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240279+0000", 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1601000000000006e-05 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.240489+0000", 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021001599999999999 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.241203+0000", 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000714674 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242405+0000", 2024-09-21T14:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00120151 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242459+0000", 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3863000000000003e-05 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242498+0000", 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9373999999999999e-05 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2723 3.5 3:a5d54d8e:::rbd_data.11212f271376.0000000000000011:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.242933+0000", 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.62518202099999998, 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00194837, 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2723 2024-09-21T14:21:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242933+0000", 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242933+0000", 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242931+0000", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242938+0000", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2729999999999999e-06 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242951+0000", 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.342e-05 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.242958+0000", 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3699999999999997e-06 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243039+0000", 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0128000000000003e-05 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243129+0000", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0612999999999995e-05 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243322+0000", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019328499999999999 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.244408+0000", 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001085787 2024-09-21T14:21:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.244779+0000", 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00037114100000000002 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.244838+0000", 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8693000000000002e-05 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.244881+0000", 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3037999999999997e-05 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2875 3.5 3:a0d5fcf9:::rbd_header.11219861e469:head [watch watch cookie 140451940735680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.508763+0000", 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.35935127700000002, 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001410754, 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2875 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508763+0000", 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508763+0000", 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508763+0000", 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508767+0000", 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3259999999999997e-06 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508777+0000", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0023e-05 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508782+0000", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.288e-06 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508822+0000", 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9206999999999999e-05 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.508902+0000", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0549000000000004e-05 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.509066+0000", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016345899999999999 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.509595+0000", 2024-09-21T14:21:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00052980099999999999 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.510062+0000", 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046635900000000001 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.510107+0000", 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5782999999999999e-05 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.510174+0000", 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6736000000000006e-05 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2994 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.755431+0000", 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.11268338, 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001420819, 2024-09-21T14:21:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2994 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755431+0000", 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755431+0000", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755430+0000", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755438+0000", 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2460000000000003e-06 2024-09-21T14:21:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755453+0000", 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4873e-05 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755461+0000", 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1389999999999995e-06 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755497+0000", 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6037000000000001e-05 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755551+0000", 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4500000000000003e-05 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.755776+0000", 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022502500000000001 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.756605+0000", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082904400000000003 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.756789+0000", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018353400000000001 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.756817+0000", 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7622000000000001e-05 2024-09-21T14:21:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.756852+0000", 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.557e-05 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2996 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.757173+0000", 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.110941594, 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021736030000000001, 2024-09-21T14:21:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2996 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757173+0000", 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757173+0000", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757172+0000", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757178+0000", 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:21:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757188+0000", 2024-09-21T14:21:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0339e-05 2024-09-21T14:21:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757194+0000", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5570000000000001e-06 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757239+0000", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4821000000000003e-05 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757334+0000", 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5095999999999999e-05 2024-09-21T14:21:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.757683+0000", 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000349105 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.759262+0000", 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015784460000000001 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.759284+0000", 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1625e-05 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.759311+0000", 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7852999999999999e-05 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.759347+0000", 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5386000000000003e-05 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3064 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.763633+0000", 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.10448176000000001, 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002209206, 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3064 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763633+0000", 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763633+0000", 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763631+0000", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763640+0000", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4970000000000001e-06 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763656+0000", 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5826000000000001e-05 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763664+0000", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763712+0000", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8353000000000003e-05 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.763813+0000", 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010075499999999999 2024-09-21T14:21:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.764113+0000", 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030006800000000001 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.764976+0000", 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00086296500000000004 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.765769+0000", 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079324199999999999 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.765813+0000", 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3676e-05 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.765842+0000", 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9142999999999999e-05 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3164 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.780251+0000", 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.087863628999999999, 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045083830000000004, 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3164 2024-09-21T14:21:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780251+0000", 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780251+0000", 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780249+0000", 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780263+0000", 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188e-05 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780285+0000", 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000001e-05 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780296+0000", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0370999999999999e-05 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780350+0000", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4317999999999997e-05 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780452+0000", 2024-09-21T14:21:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102198 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.780870+0000", 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000417685 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.781955+0000", 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010850440000000001 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784642+0000", 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026865790000000001 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784692+0000", 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0275999999999998e-05 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784760+0000", 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7602999999999996e-05 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3165 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.782252+0000", 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.085863002999999993, 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028130680000000002, 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3165 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782252+0000", 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782252+0000", 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782251+0000", 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782256+0000", 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782270+0000", 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3275000000000001e-05 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782277+0000", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5240000000000003e-06 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782337+0000", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0343000000000001e-05 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782433+0000", 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.556e-05 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.782810+0000", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00037689599999999999 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784825+0000", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002014708 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784975+0000", 2024-09-21T14:21:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000150598 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785026+0000", 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1014999999999999e-05 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785065+0000", 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.858e-05 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3166 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.784027+0000", 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.084087563000000004, 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003745483, 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3166 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784027+0000", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784027+0000", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784026+0000", 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784032+0000", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3480000000000003e-06 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784042+0000", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0468999999999999e-05 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784047+0000", 2024-09-21T14:21:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1039999999999998e-06 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784078+0000", 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0348000000000001e-05 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784137+0000", 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9278999999999999e-05 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784336+0000", 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019928599999999999 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.787665+0000", 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033284959999999998 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.787683+0000", 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8819000000000001e-05 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.787734+0000", 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0351e-05 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.787773+0000", 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8829999999999999e-05 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3167 3.5 3:a0fa18b1:::rbd_data.1121cf46bdbd.00000000000000e7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.083250010999999999, 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053239749999999999, 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3167 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784864+0000", 2024-09-21T14:21:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784870+0000", 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3620000000000002e-06 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784883+0000", 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2839e-05 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784890+0000", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5070000000000003e-06 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784942+0000", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1490000000000003e-05 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785057+0000", 2024-09-21T14:21:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115359 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785161+0000", 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104171 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785979+0000", 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081793100000000004 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790095+0000", 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041157499999999996 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790147+0000", 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.223e-05 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790189+0000", 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1365999999999997e-05 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3171 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.791494+0000", 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.076620619000000001, 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001627987, 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3171 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791494+0000", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791494+0000", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791493+0000", 2024-09-21T14:21:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791500+0000", 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0469999999999996e-06 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791514+0000", 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4168999999999999e-05 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791521+0000", 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3100000000000003e-06 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791557+0000", 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6089e-05 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791614+0000", 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6440000000000002e-05 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.791774+0000", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015999300000000001 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.792913+0000", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001139508 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.793072+0000", 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015839900000000001 2024-09-21T14:21:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.793101+0000", 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9665000000000001e-05 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.793122+0000", 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.073e-05 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3179 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.801894+0000", 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.066221226999999994, 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001420852, 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3179 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801894+0000", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801894+0000", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801891+0000", 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:21:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801907+0000", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.664e-05 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801937+0000", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9896999999999999e-05 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.801950+0000", 2024-09-21T14:21:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3033e-05 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.802017+0000", 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6335999999999996e-05 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.802091+0000", 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4221000000000006e-05 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.802288+0000", 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019709400000000001 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.802972+0000", 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068353800000000003 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.803258+0000", 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000286698 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.803288+0000", 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0074999999999999e-05 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.803314+0000", 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6057000000000001e-05 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3188 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.807155+0000", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.060959309000000003, 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037309169999999998, 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3188 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807155+0000", 2024-09-21T14:21:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807155+0000", 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807154+0000", 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807160+0000", 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2110000000000003e-06 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807172+0000", 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.117e-05 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807177+0000", 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.626e-06 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807217+0000", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9342999999999999e-05 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807275+0000", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8551999999999998e-05 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807478+0000", 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202673 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810482+0000", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003004142 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810822+0000", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000339902 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810852+0000", 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9734999999999999e-05 2024-09-21T14:21:24.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810886+0000", 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4752000000000002e-05 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3189 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.807188+0000", 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.060926611999999998, 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040123629999999997, 2024-09-21T14:21:24.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3189 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807188+0000", 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807188+0000", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807188+0000", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807191+0000", 2024-09-21T14:21:24.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807197+0000", 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3990000000000001e-06 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807202+0000", 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0590000000000002e-06 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807791+0000", 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058885000000000003 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807880+0000", 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8635999999999996e-05 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.808137+0000", 2024-09-21T14:21:24.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000256748 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810901+0000", 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002764449 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811141+0000", 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023982100000000001 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811166+0000", 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.546e-05 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811201+0000", 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4048999999999999e-05 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.978 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3190 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.807207+0000", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.060907913000000001, 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040887320000000003, 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3190 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:24.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807207+0000", 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807207+0000", 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807207+0000", 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807210+0000", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9960000000000002e-06 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807216+0000", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3249999999999996e-06 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.807224+0000", 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9039999999999992e-06 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.808445+0000", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001221544 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.808493+0000", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7593000000000003e-05 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.808681+0000", 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018773600000000001 2024-09-21T14:21:24.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.810906+0000", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022249779999999999 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811230+0000", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032479399999999999 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811265+0000", 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4662000000000001e-05 2024-09-21T14:21:24.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.811296+0000", 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0525999999999999e-05 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:24.984 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:24.986 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:24.986 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:24.988 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:24.989 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T14:21:24.421166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95445985,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":978,"num_read_kb":1172,"num_write":314,"num_write_kb":3395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218220,"kb_used_data":2040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756500,"statfs":{"total":773094113280,"available":772870656000,"internally_reserved":0,"allocated":2088960,"data_stored":1925383,"data_compressed":66423,"data_compressed_allocated":94208,"data_compressed_original":1179648,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000562"},"pg_stats":[{"pgid":"3.1f","version":"22'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034921+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:22.034921+0000","last_peered":"2024-09-21T14:21:22.034921+0000","last_clean":"2024-09-21T14:21:22.034921+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:22.034921+0000","last_undegraded":"2024-09-21T14:21:22.034921+0000","last_fullsized":"2024-09-21T14:21:22.034921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"23'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.294461+0000","last_change":"2024-09-21T14:21:20.149718+0000","last_active":"2024-09-21T14:21:22.294461+0000","last_peered":"2024-09-21T14:21:22.294461+0000","last_clean":"2024-09-21T14:21:22.294461+0000","last_became_active":"2024-09-21T14:21:18.152993+0000","last_became_peered":"2024-09-21T14:21:18.152993+0000","last_unstale":"2024-09-21T14:21:22.294461+0000","last_undegraded":"2024-09-21T14:21:22.294461+0000","last_fullsized":"2024-09-21T14:21:22.294461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:18.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":3207094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":31,"num_write":1,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152252+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:21.152252+0000","last_peered":"2024-09-21T14:21:21.152252+0000","last_clean":"2024-09-21T14:21:21.152252+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:21.152252+0000","last_undegraded":"2024-09-21T14:21:21.152252+0000","last_fullsized":"2024-09-21T14:21:21.152252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"24'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.313966+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:22.313966+0000","last_peered":"2024-09-21T14:21:22.313966+0000","last_clean":"2024-09-21T14:21:22.313966+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:22.313966+0000","last_undegraded":"2024-09-21T14:21:22.313966+0000","last_fullsized":"2024-09-21T14:21:22.313966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":3995317,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":18,"num_write":3,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975561+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:21.975561+0000","last_peered":"2024-09-21T14:21:21.975561+0000","last_clean":"2024-09-21T14:21:21.975561+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:21.975561+0000","last_undegraded":"2024-09-21T14:21:21.975561+0000","last_fullsized":"2024-09-21T14:21:21.975561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"22'2","reported_seq":47,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.382719+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:22.382719+0000","last_peered":"2024-09-21T14:21:22.382719+0000","last_clean":"2024-09-21T14:21:22.382719+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:22.382719+0000","last_undegraded":"2024-09-21T14:21:22.382719+0000","last_fullsized":"2024-09-21T14:21:22.382719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"24'4","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418472+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:22.418472+0000","last_peered":"2024-09-21T14:21:22.418472+0000","last_clean":"2024-09-21T14:21:22.418472+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:22.418472+0000","last_undegraded":"2024-09-21T14:21:22.418472+0000","last_fullsized":"2024-09-21T14:21:22.418472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":7534650,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":1,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.409252+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:22.409252+0000","last_peered":"2024-09-21T14:21:22.409252+0000","last_clean":"2024-09-21T14:21:22.409252+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:22.409252+0000","last_undegraded":"2024-09-21T14:21:22.409252+0000","last_fullsized":"2024-09-21T14:21:22.409252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":13184888,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":1,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034956+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:22.034956+0000","last_peered":"2024-09-21T14:21:22.034956+0000","last_clean":"2024-09-21T14:21:22.034956+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:22.034956+0000","last_undegraded":"2024-09-21T14:21:22.034956+0000","last_fullsized":"2024-09-21T14:21:22.034956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151529+0000","last_change":"2024-09-21T14:21:20.755882+0000","last_active":"2024-09-21T14:21:21.151529+0000","last_peered":"2024-09-21T14:21:21.151529+0000","last_clean":"2024-09-21T14:21:21.151529+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.151529+0000","last_undegraded":"2024-09-21T14:21:21.151529+0000","last_fullsized":"2024-09-21T14:21:21.151529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:17.531857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.256146+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:21.256146+0000","last_peered":"2024-09-21T14:21:21.256146+0000","last_clean":"2024-09-21T14:21:21.256146+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:21.256146+0000","last_undegraded":"2024-09-21T14:21:21.256146+0000","last_fullsized":"2024-09-21T14:21:21.256146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152175+0000","last_change":"2024-09-21T14:21:20.161641+0000","last_active":"2024-09-21T14:21:21.152175+0000","last_peered":"2024-09-21T14:21:21.152175+0000","last_clean":"2024-09-21T14:21:21.152175+0000","last_became_active":"2024-09-21T14:21:18.152393+0000","last_became_peered":"2024-09-21T14:21:18.152393+0000","last_unstale":"2024-09-21T14:21:21.152175+0000","last_undegraded":"2024-09-21T14:21:21.152175+0000","last_fullsized":"2024-09-21T14:21:21.152175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:55.413999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.877129+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:21.877129+0000","last_peered":"2024-09-21T14:21:21.877129+0000","last_clean":"2024-09-21T14:21:21.877129+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:21.877129+0000","last_undegraded":"2024-09-21T14:21:21.877129+0000","last_fullsized":"2024-09-21T14:21:21.877129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'16","reported_seq":96,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.424523+0000","last_change":"2024-09-21T14:21:20.755817+0000","last_active":"2024-09-21T14:21:22.424523+0000","last_peered":"2024-09-21T14:21:22.424523+0000","last_clean":"2024-09-21T14:21:22.424523+0000","last_became_active":"2024-09-21T14:21:18.150612+0000","last_became_peered":"2024-09-21T14:21:18.150612+0000","last_unstale":"2024-09-21T14:21:22.424523+0000","last_undegraded":"2024-09-21T14:21:22.424523+0000","last_fullsized":"2024-09-21T14:21:22.424523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:50.934485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":180,"num_read_kb":154,"num_write":15,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152523+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:21.152523+0000","last_peered":"2024-09-21T14:21:21.152523+0000","last_clean":"2024-09-21T14:21:21.152523+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:21.152523+0000","last_undegraded":"2024-09-21T14:21:21.152523+0000","last_fullsized":"2024-09-21T14:21:21.152523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.927326+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:21.927326+0000","last_peered":"2024-09-21T14:21:21.927326+0000","last_clean":"2024-09-21T14:21:21.927326+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:21.927326+0000","last_undegraded":"2024-09-21T14:21:21.927326+0000","last_fullsized":"2024-09-21T14:21:21.927326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151554+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:21.151554+0000","last_peered":"2024-09-21T14:21:21.151554+0000","last_clean":"2024-09-21T14:21:21.151554+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:21.151554+0000","last_undegraded":"2024-09-21T14:21:21.151554+0000","last_fullsized":"2024-09-21T14:21:21.151554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875923+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:21.875923+0000","last_peered":"2024-09-21T14:21:21.875923+0000","last_clean":"2024-09-21T14:21:21.875923+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:21.875923+0000","last_undegraded":"2024-09-21T14:21:21.875923+0000","last_fullsized":"2024-09-21T14:21:21.875923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.947981+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:21.947981+0000","last_peered":"2024-09-21T14:21:21.947981+0000","last_clean":"2024-09-21T14:21:21.947981+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:21.947981+0000","last_undegraded":"2024-09-21T14:21:21.947981+0000","last_fullsized":"2024-09-21T14:21:21.947981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'6","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.416170+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:22.416170+0000","last_peered":"2024-09-21T14:21:22.416170+0000","last_clean":"2024-09-21T14:21:22.416170+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:22.416170+0000","last_undegraded":"2024-09-21T14:21:22.416170+0000","last_fullsized":"2024-09-21T14:21:22.416170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":15209098,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":125,"num_write":3,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975446+0000","last_change":"2024-09-21T14:21:20.755906+0000","last_active":"2024-09-21T14:21:21.975446+0000","last_peered":"2024-09-21T14:21:21.975446+0000","last_clean":"2024-09-21T14:21:21.975446+0000","last_became_active":"2024-09-21T14:21:18.151157+0000","last_became_peered":"2024-09-21T14:21:18.151157+0000","last_unstale":"2024-09-21T14:21:21.975446+0000","last_undegraded":"2024-09-21T14:21:21.975446+0000","last_fullsized":"2024-09-21T14:21:21.975446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:25.967224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152075+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:21.152075+0000","last_peered":"2024-09-21T14:21:21.152075+0000","last_clean":"2024-09-21T14:21:21.152075+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:21.152075+0000","last_undegraded":"2024-09-21T14:21:21.152075+0000","last_fullsized":"2024-09-21T14:21:21.152075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875956+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:21.875956+0000","last_peered":"2024-09-21T14:21:21.875956+0000","last_clean":"2024-09-21T14:21:21.875956+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:21.875956+0000","last_undegraded":"2024-09-21T14:21:21.875956+0000","last_fullsized":"2024-09-21T14:21:21.875956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034849+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:22.034849+0000","last_peered":"2024-09-21T14:21:22.034849+0000","last_clean":"2024-09-21T14:21:22.034849+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:22.034849+0000","last_undegraded":"2024-09-21T14:21:22.034849+0000","last_fullsized":"2024-09-21T14:21:22.034849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.255796+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:21.255796+0000","last_peered":"2024-09-21T14:21:21.255796+0000","last_clean":"2024-09-21T14:21:21.255796+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:21.255796+0000","last_undegraded":"2024-09-21T14:21:21.255796+0000","last_fullsized":"2024-09-21T14:21:21.255796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"24'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.182951+0000","last_change":"2024-09-21T14:21:20.151933+0000","last_active":"2024-09-21T14:21:22.182951+0000","last_peered":"2024-09-21T14:21:22.182951+0000","last_clean":"2024-09-21T14:21:22.182951+0000","last_became_active":"2024-09-21T14:21:18.147902+0000","last_became_peered":"2024-09-21T14:21:18.147902+0000","last_unstale":"2024-09-21T14:21:22.182951+0000","last_undegraded":"2024-09-21T14:21:22.182951+0000","last_fullsized":"2024-09-21T14:21:22.182951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:09.190450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":41,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152478+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:21.152478+0000","last_peered":"2024-09-21T14:21:21.152478+0000","last_clean":"2024-09-21T14:21:21.152478+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:21.152478+0000","last_undegraded":"2024-09-21T14:21:21.152478+0000","last_fullsized":"2024-09-21T14:21:21.152478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"24'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.355019+0000","last_change":"2024-09-21T14:21:20.145954+0000","last_active":"2024-09-21T14:21:22.355019+0000","last_peered":"2024-09-21T14:21:22.355019+0000","last_clean":"2024-09-21T14:21:22.355019+0000","last_became_active":"2024-09-21T14:21:18.152479+0000","last_became_peered":"2024-09-21T14:21:18.152479+0000","last_unstale":"2024-09-21T14:21:22.355019+0000","last_undegraded":"2024-09-21T14:21:22.355019+0000","last_fullsized":"2024-09-21T14:21:22.355019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:27.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":16777232,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":6,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875848+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:21.875848+0000","last_peered":"2024-09-21T14:21:21.875848+0000","last_clean":"2024-09-21T14:21:21.875848+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:21.875848+0000","last_undegraded":"2024-09-21T14:21:21.875848+0000","last_fullsized":"2024-09-21T14:21:21.875848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"24'37","reported_seq":59,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418646+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:22.418646+0000","last_peered":"2024-09-21T14:21:22.418646+0000","last_clean":"2024-09-21T14:21:22.418646+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:22.418646+0000","last_undegraded":"2024-09-21T14:21:22.418646+0000","last_fullsized":"2024-09-21T14:21:22.418646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":6837908,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":194,"num_write":53,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"24'17","reported_seq":35,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.379636+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:22.379636+0000","last_peered":"2024-09-21T14:21:22.379636+0000","last_clean":"2024-09-21T14:21:22.379636+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:22.379636+0000","last_undegraded":"2024-09-21T14:21:22.379636+0000","last_fullsized":"2024-09-21T14:21:22.379636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":36,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":51,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"24'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.354567+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:22.354567+0000","last_peered":"2024-09-21T14:21:22.354567+0000","last_clean":"2024-09-21T14:21:22.354567+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:22.354567+0000","last_undegraded":"2024-09-21T14:21:22.354567+0000","last_fullsized":"2024-09-21T14:21:22.354567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":16167918,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":35,"num_write":3,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"23'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.345351+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:22.345351+0000","last_peered":"2024-09-21T14:21:22.345351+0000","last_clean":"2024-09-21T14:21:22.345351+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:22.345351+0000","last_undegraded":"2024-09-21T14:21:22.345351+0000","last_fullsized":"2024-09-21T14:21:22.345351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":1593100,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152309+0000","last_change":"2024-09-21T14:21:20.162015+0000","last_active":"2024-09-21T14:21:21.152309+0000","last_peered":"2024-09-21T14:21:21.152309+0000","last_clean":"2024-09-21T14:21:21.152309+0000","last_became_active":"2024-09-21T14:21:18.143760+0000","last_became_peered":"2024-09-21T14:21:18.143760+0000","last_unstale":"2024-09-21T14:21:21.152309+0000","last_undegraded":"2024-09-21T14:21:21.152309+0000","last_fullsized":"2024-09-21T14:21:21.152309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"23'4","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.015396+0000","last_change":"2024-09-21T14:21:20.756021+0000","last_active":"2024-09-21T14:21:22.015396+0000","last_peered":"2024-09-21T14:21:22.015396+0000","last_clean":"2024-09-21T14:21:22.015396+0000","last_became_active":"2024-09-21T14:21:18.152859+0000","last_became_peered":"2024-09-21T14:21:18.152859+0000","last_unstale":"2024-09-21T14:21:22.015396+0000","last_undegraded":"2024-09-21T14:21:22.015396+0000","last_fullsized":"2024-09-21T14:21:22.015396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.438917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"24'21","reported_seq":163,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.285127+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:22.285127+0000","last_peered":"2024-09-21T14:21:22.285127+0000","last_clean":"2024-09-21T14:21:22.285127+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:22.285127+0000","last_undegraded":"2024-09-21T14:21:22.285127+0000","last_fullsized":"2024-09-21T14:21:22.285127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":359,"num_read_kb":311,"num_write":19,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.426849+0000","last_change":"2024-09-21T14:21:20.160536+0000","last_active":"2024-09-21T14:21:22.426849+0000","last_peered":"2024-09-21T14:21:22.426849+0000","last_clean":"2024-09-21T14:21:22.426849+0000","last_became_active":"2024-09-21T14:21:18.151161+0000","last_became_peered":"2024-09-21T14:21:18.151161+0000","last_unstale":"2024-09-21T14:21:22.426849+0000","last_undegraded":"2024-09-21T14:21:22.426849+0000","last_fullsized":"2024-09-21T14:21:22.426849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:48.604891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":10348281,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":119,"num_write":8,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.958641+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:21.958641+0000","last_peered":"2024-09-21T14:21:21.958641+0000","last_clean":"2024-09-21T14:21:21.958641+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.958641+0000","last_undegraded":"2024-09-21T14:21:21.958641+0000","last_fullsized":"2024-09-21T14:21:21.958641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152151+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:21.152151+0000","last_peered":"2024-09-21T14:21:21.152151+0000","last_clean":"2024-09-21T14:21:21.152151+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:21.152151+0000","last_undegraded":"2024-09-21T14:21:21.152151+0000","last_fullsized":"2024-09-21T14:21:21.152151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":527422,"data_compressed":18878,"data_compressed_allocated":49152,"data_compressed_original":385024,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":83424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":92810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":672841,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":83473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":679631,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":89233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":134719,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":89252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":61208,"data_compressed":2228,"data_compressed_allocated":4096,"data_compressed_original":45056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":145636,"data_compressed":6414,"data_compressed_allocated":16384,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":66052,"data_compressed":3021,"data_compressed_allocated":8192,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":162317,"data_compressed":6046,"data_compressed_allocated":16384,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":31721,"data_compressed":1169,"data_compressed_allocated":4096,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":10839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:24.991 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:21:24.991 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:24.986+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:24.991 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:21:24.992 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:21:24.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:21:25.033 INFO:teuthology.orchestra.run.smithi073.stdout:190 write 0xebb3966 thru 0xebb3f79 (0x614 bytes) 2024-09-21T14:21:25.037 INFO:teuthology.orchestra.run.smithi073.stdout:191 write 0x2fc046c thru 0x2fcf253 (0xede8 bytes) 2024-09-21T14:21:25.059 INFO:teuthology.orchestra.run.smithi073.stdout:192 trunc from 0xebb3f7a to 0x6c76de2 2024-09-21T14:21:25.084 INFO:teuthology.orchestra.run.smithi073.stdout:194 read 0x2dedc92 thru 0x2df32ad (0x561c bytes) 2024-09-21T14:21:25.085 INFO:teuthology.orchestra.run.smithi073.stdout:195 punch from 0x8a5176 to 0x8adc93, (0x8b1d bytes) 2024-09-21T14:21:25.086 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:25.086+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:25.090 INFO:teuthology.orchestra.run.smithi073.stdout:196 write 0x7185f30 thru 0x71909d1 (0xaaa2 bytes) 2024-09-21T14:21:25.095 INFO:teuthology.orchestra.run.smithi073.stdout:199 writesame 0x472ac2b thru 0x4736aa0 data_size 0x1(0xbe76 bytes) 2024-09-21T14:21:25.125 INFO:teuthology.orchestra.run.smithi073.stdout:200 trunc from 0x71909d2 to 0x51e09c 2024-09-21T14:21:25.158 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.158 INFO:teuthology.orchestra.run.smithi073.stdout:202 read 0x1b5ac9 thru 0x1c2177 (0xc6af bytes) 2024-09-21T14:21:25.159 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.159 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:25.159 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:25.159 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:25.160 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:25.166 INFO:teuthology.orchestra.run.smithi073.stdout:204 write 0x1063e45 thru 0x106d845 (0x9a01 bytes) 2024-09-21T14:21:25.170 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:25.176 INFO:teuthology.orchestra.run.smithi073.stdout:205 read 0x1a7016 thru 0x1adcce (0x6cb9 bytes) 2024-09-21T14:21:25.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:21:25.187 INFO:teuthology.orchestra.run.smithi073.stdout:206 writesame 0x3fbc28d thru 0x3fc347a data_size 0x3(0x71ee bytes) 2024-09-21T14:21:25.187 INFO:teuthology.orchestra.run.smithi073.stdout:208 read 0x1eb7601 thru 0x1ec621a (0xec1a bytes) 2024-09-21T14:21:25.187 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:25.187+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:25.208 INFO:teuthology.orchestra.run.smithi073.stdout:209 writesame 0xeb9f400 thru 0xeba4194 data_size 0x1(0x4d95 bytes) 2024-09-21T14:21:25.208 INFO:teuthology.orchestra.run.smithi073.stdout:210 punch from 0x347599e to 0x3476b87, (0x11e9 bytes) 2024-09-21T14:21:25.220 INFO:teuthology.orchestra.run.smithi073.stdout:211 trunc from 0xeba4195 to 0x55afb0d 2024-09-21T14:21:25.231 INFO:teuthology.orchestra.run.smithi073.stdout:212 trunc from 0x55afb0d to 0x861c7d 2024-09-21T14:21:25.242 INFO:teuthology.orchestra.run.smithi073.stdout:213 read 0x2a68d2 thru 0x2b5e50 (0xf57f bytes) 2024-09-21T14:21:25.243 INFO:teuthology.orchestra.run.smithi073.stdout:214 read 0x12b6c4 thru 0x133b97 (0x84d4 bytes) 2024-09-21T14:21:25.253 INFO:teuthology.orchestra.run.smithi073.stdout:215 trunc from 0x861c7d to 0x8785e21 2024-09-21T14:21:25.258 INFO:teuthology.orchestra.run.smithi073.stdout:216 writesame 0x7dbf14d thru 0x7dcbb84 data_size 0x24(0xca38 bytes) 2024-09-21T14:21:25.266 INFO:teuthology.orchestra.run.smithi073.stdout:217 read 0x107600a thru 0x10813e3 (0xb3da bytes) 2024-09-21T14:21:25.266 INFO:teuthology.orchestra.run.smithi073.stdout:220 punch from 0x2c982e4 to 0x2c9d4f1, (0x520d bytes) 2024-09-21T14:21:25.267 INFO:teuthology.orchestra.run.smithi073.stdout:226 punch from 0x870de51 to 0x870f753, (0x1902 bytes) 2024-09-21T14:21:25.269 INFO:teuthology.orchestra.run.smithi073.stdout:227 trunc from 0x8785e21 to 0x66b69e9 2024-09-21T14:21:25.281 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.281 INFO:teuthology.orchestra.run.smithi073.stdout:229 read 0x3268452 thru 0x3272fb6 (0xab65 bytes) 2024-09-21T14:21:25.281 INFO:teuthology.orchestra.run.smithi073.stdout:230 punch from 0x34af081 to 0x34b68c4, (0x7843 bytes) 2024-09-21T14:21:25.281 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.287 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:25.286+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:25.293 INFO:teuthology.orchestra.run.smithi073.stdout:233 writesame 0xc427068 thru 0xc42a3c2 data_size 0x1(0x335b bytes) 2024-09-21T14:21:25.294 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.294 INFO:teuthology.orchestra.run.smithi073.stdout:237 punch from 0x4946980 to 0x4947be4, (0x1264 bytes) 2024-09-21T14:21:25.298 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:25.298 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:25.298 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:25.298 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:25.299 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:25.299 INFO:teuthology.orchestra.run.smithi073.stdout:239 trunc from 0xc42a3c3 to 0x810aedd 2024-09-21T14:21:25.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:25.318 INFO:teuthology.orchestra.run.smithi073.stdout:240 write 0xe4be6a4 thru 0xe4c987b (0xb1d8 bytes) 2024-09-21T14:21:25.325 INFO:teuthology.orchestra.run.smithi073.stdout:241 punch from 0xbaf0729 to 0xbaff0cf, (0xe9a6 bytes) 2024-09-21T14:21:25.325 INFO:teuthology.orchestra.run.smithi073.stdout:243 punch from 0x7706cb1 to 0x7707647, (0x996 bytes) 2024-09-21T14:21:25.325 INFO:teuthology.orchestra.run.smithi073.stdout:247 write 0x6c5d9ef thru 0x6c60442 (0x2a54 bytes) 2024-09-21T14:21:25.331 INFO:teuthology.orchestra.run.smithi073.stdout:250 write 0x7886f47 thru 0x788e127 (0x71e1 bytes) 2024-09-21T14:21:25.339 INFO:teuthology.orchestra.run.smithi073.stdout:251 read 0x82a077f thru 0x82a1ba1 (0x1423 bytes) 2024-09-21T14:21:25.339 INFO:teuthology.orchestra.run.smithi073.stdout:252 punch from 0x57ad16b to 0x57bc1b3, (0xf048 bytes) 2024-09-21T14:21:25.339 INFO:teuthology.orchestra.run.smithi073.stdout:254 read 0x216cd31 thru 0x216d714 (0x9e4 bytes) 2024-09-21T14:21:25.339 INFO:teuthology.orchestra.run.smithi073.stdout:255 write 0xbf941c4 thru 0xbf9634a (0x2187 bytes) 2024-09-21T14:21:25.342 INFO:teuthology.orchestra.run.smithi073.stdout:257 read 0xddb3d5d thru 0xddb57fc (0x1aa0 bytes) 2024-09-21T14:21:25.342 INFO:teuthology.orchestra.run.smithi073.stdout:258 read 0x1e3f925 thru 0x1e4f1f9 (0xf8d5 bytes) 2024-09-21T14:21:25.342 INFO:teuthology.orchestra.run.smithi073.stdout:260 read 0xcdb3c2c thru 0xcdb61dd (0x25b2 bytes) 2024-09-21T14:21:25.342 INFO:teuthology.orchestra.run.smithi073.stdout:261 writesame 0x94c0b33 thru 0x94c9ff9 data_size 0x1(0x94c7 bytes) 2024-09-21T14:21:25.360 INFO:teuthology.orchestra.run.smithi073.stdout:263 punch from 0x9bfdbef to 0x9bfece5, (0x10f6 bytes) 2024-09-21T14:21:25.362 INFO:teuthology.orchestra.run.smithi073.stdout:264 trunc from 0xe4c987c to 0xc915fda 2024-09-21T14:21:25.373 INFO:teuthology.orchestra.run.smithi073.stdout:266 read 0x792c59a thru 0x7936a17 (0xa47e bytes) 2024-09-21T14:21:25.374 INFO:teuthology.orchestra.run.smithi073.stdout:267 punch from 0x2129285 to 0x212cf7f, (0x3cfa bytes) 2024-09-21T14:21:25.374 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.374 INFO:teuthology.orchestra.run.smithi073.stdout:270 write 0xc02aac0 thru 0xc035d90 (0xb2d1 bytes) 2024-09-21T14:21:25.380 INFO:teuthology.orchestra.run.smithi073.stdout:272 write 0xacfa029 thru 0xad09e10 (0xfde8 bytes) 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 26, 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:21:24.164258+0000", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:20:58.662769+0000", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5804032, 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:21:25.382 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:21:25.383 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "jewel", 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:25.384 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2, 2024-09-21T14:21:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 1, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:21:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:21:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 1, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:21:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 6, 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 26, 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.75, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.75, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1, 2024-09-21T14:21:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.75, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.75, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 1, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:21:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:21:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:21:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:21:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:21:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [], 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:21:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [ 2024-09-21T14:21:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "snaps": [ 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "begin": 2, 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "length": 1 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:21:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.465 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:25.466 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.467 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.467 INFO:teuthology.orchestra.run.smithi073.stdout:275 writesame 0xe2bd0fc thru 0xe2cb11d data_size 0x6(0xe022 bytes) 2024-09-21T14:21:25.467 INFO:teuthology.orchestra.run.smithi073.stdout:277 trunc from 0xe2cb11e to 0x9c07669 2024-09-21T14:21:25.468 INFO:tasks.thrashosds.thrasher:j is {'epoch': 26, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:21:24.164258+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:20:58.662769+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 26, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:21:25.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:25.469 INFO:teuthology.orchestra.run.smithi073.stdout:278 trunc from 0x9c07669 to 0x3fc7cde 2024-09-21T14:21:25.469 INFO:teuthology.orchestra.run.smithi073.stdout:280 write 0x8a84ae3 thru 0x8a90989 (0xbea7 bytes) 2024-09-21T14:21:25.469 INFO:teuthology.orchestra.run.smithi073.stdout:281 trunc from 0x8a9098a to 0x85ef5df 2024-09-21T14:21:25.469 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:25.466+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:25.470 INFO:teuthology.orchestra.run.smithi073.stdout:282 trunc from 0x85ef5df to 0x2f8aeab 2024-09-21T14:21:25.485 INFO:teuthology.orchestra.run.smithi073.stdout:284 write 0x6fc652c thru 0x6fca11b (0x3bf0 bytes) 2024-09-21T14:21:25.489 INFO:teuthology.orchestra.run.smithi073.stdout:285 punch from 0x59c5e00 to 0x59d4c59, (0xee59 bytes) 2024-09-21T14:21:25.489 INFO:teuthology.orchestra.run.smithi073.stdout:286 read 0x6af5acb thru 0x6afb358 (0x588e bytes) 2024-09-21T14:21:25.489 INFO:teuthology.orchestra.run.smithi073.stdout:287 read 0x21b7bc thru 0x220c64 (0x54a9 bytes) 2024-09-21T14:21:25.489 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.494 INFO:teuthology.orchestra.run.smithi073.stdout:289 trunc from 0x6fca11c to 0x2bb5dfb 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:25.496 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.952683991000001, 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:25.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:25.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2556 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.256329947, 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014254843, 2024-09-21T14:21:25.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2556 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238295+0000", 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238301+0000", 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7209999999999999e-06 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238303+0000", 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3999999999999999e-06 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252353+0000", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014049918999999999 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252438+0000", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5140000000000001e-05 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252548+0000", 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110009 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.508 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2557 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18153472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2563169840000001, 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014393985, 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2557 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238308+0000", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8339999999999999e-06 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238313+0000", 2024-09-21T14:21:25.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1880000000000001e-06 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238317+0000", 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0910000000000003e-06 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252584+0000", 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014267334 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252642+0000", 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8162000000000003e-05 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252700+0000", 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7827e-05 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2558 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18481152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2563027660000001, 2024-09-21T14:21:25.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014618236999999999, 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2558 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238320+0000", 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238322+0000", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8700000000000001e-06 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238327+0000", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238331+0000", 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8210000000000003e-06 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252746+0000", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014415156 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252830+0000", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3974000000000006e-05 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252939+0000", 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108448 2024-09-21T14:21:25.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2559 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18808832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2562881969999999, 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014758411000000001, 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2559 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:25.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238337+0000", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7290000000000001e-06 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238342+0000", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4709999999999998e-06 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238346+0000", 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9310000000000001e-06 2024-09-21T14:21:25.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252963+0000", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014616878999999999 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.253026+0000", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3516000000000006e-05 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.253094+0000", 2024-09-21T14:21:25.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7224000000000006e-05 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2742 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2513209810000001, 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014172158000000001, 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2742 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243305+0000", 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.779e-06 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243311+0000", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.19e-06 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243316+0000", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4890000000000003e-06 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257345+0000", 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014028491000000001 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257410+0000", 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5575999999999997e-05 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257475+0000", 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4015000000000003e-05 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2743 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2513025360000001, 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014360362999999999, 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2743 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243320+0000", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243323+0000", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.492e-06 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243333+0000", 2024-09-21T14:21:25.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5990000000000008e-06 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243337+0000", 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.42e-06 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257514+0000", 2024-09-21T14:21:25.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014177286000000001 2024-09-21T14:21:25.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257583+0000", 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9256000000000002e-05 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257681+0000", 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7678999999999998e-05 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2744 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2512821249999999, 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014566601, 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.535 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2744 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243343+0000", 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4449999999999999e-06 2024-09-21T14:21:25.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243350+0000", 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8929999999999999e-06 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243355+0000", 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4789999999999998e-06 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257726+0000", 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014371754 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257815+0000", 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9054000000000005e-05 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257908+0000", 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2342999999999993e-05 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2745 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.251263934, 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014728415, 2024-09-21T14:21:25.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2745 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243361+0000", 2024-09-21T14:21:25.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3379999999999999e-06 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243368+0000", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9670000000000004e-06 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243373+0000", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0150000000000003e-06 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257934+0000", 2024-09-21T14:21:25.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014560518 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258022+0000", 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8175000000000002e-05 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258088+0000", 2024-09-21T14:21:25.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5741999999999995e-05 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2746 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2512455840000001, 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014896428, 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.545 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2746 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243377+0000", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243399+0000", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1429000000000001e-05 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243406+0000", 2024-09-21T14:21:25.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8510000000000001e-06 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243411+0000", 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1669999999999998e-06 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258108+0000", 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014697306 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258185+0000", 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6841000000000005e-05 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258274+0000", 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9184999999999994e-05 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2747 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2512076059999999, 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015034008999999999, 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2747 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243415+0000", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243418+0000", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243424+0000", 2024-09-21T14:21:25.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7970000000000003e-06 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243446+0000", 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2084000000000001e-05 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258310+0000", 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014864007 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258389+0000", 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9016000000000004e-05 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258450+0000", 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0309000000000001e-05 2024-09-21T14:21:25.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2748 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.251172027, 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015218272, 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2748 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243454+0000", 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7310000000000002e-06 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243460+0000", 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243464+0000", 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4039999999999997e-06 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258488+0000", 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015024084 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258569+0000", 2024-09-21T14:21:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1258e-05 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258670+0000", 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100163 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2749 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.25115504, 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015336407, 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2749 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243474+0000", 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9580000000000004e-06 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243481+0000", 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6719999999999998e-06 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243494+0000", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2979999999999999e-05 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258712+0000", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015217965 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258754+0000", 2024-09-21T14:21:25.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2005999999999997e-05 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258805+0000", 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1184000000000003e-05 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2750 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.565 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2510923110000001, 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015489749000000001, 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2750 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243530+0000", 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243535+0000", 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.42e-06 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243543+0000", 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1810000000000001e-06 2024-09-21T14:21:25.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243549+0000", 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2090000000000002e-06 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258838+0000", 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015288652999999999 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258914+0000", 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5971999999999997e-05 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259021+0000", 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010700399999999999 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2751 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16908288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.251069261, 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015592231, 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2751 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243556+0000", 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4870000000000001e-06 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243562+0000", 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7019999999999998e-06 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243569+0000", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5900000000000002e-06 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259045+0000", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015475668 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259095+0000", 2024-09-21T14:21:25.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9656e-05 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259146+0000", 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1542999999999998e-05 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.575 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2752 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17235968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.251049654, 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015780444000000001, 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2752 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243573+0000", 2024-09-21T14:21:25.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243576+0000", 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.429e-06 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243582+0000", 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8499999999999999e-06 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243586+0000", 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6489999999999999e-06 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259178+0000", 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015592079999999999 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259255+0000", 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6403999999999996e-05 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259354+0000", 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9415999999999998e-05 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2753 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.251033171, 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015953208, 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2753 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243592+0000", 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8810000000000001e-06 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243597+0000", 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6110000000000001e-06 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243602+0000", 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.318e-06 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259382+0000", 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01578061 2024-09-21T14:21:25.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259455+0000", 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2459000000000005e-05 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259543+0000", 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8617999999999998e-05 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2754 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17891328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2510170899999999, 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016076594999999999, 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2754 2024-09-21T14:21:25.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243608+0000", 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0269999999999998e-06 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243613+0000", 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4999999999999999e-06 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243618+0000", 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4490000000000001e-06 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259579+0000", 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015961177 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259624+0000", 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5064000000000001e-05 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259683+0000", 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8666999999999999e-05 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2755 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18219008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.2510013609999999, 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016281739999999999, 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2755 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243624+0000", 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9939999999999999e-06 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243629+0000", 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5759999999999996e-06 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243634+0000", 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4270000000000001e-06 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259733+0000", 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016098918 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259808+0000", 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5228999999999994e-05 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259904+0000", 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5873999999999997e-05 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2756 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18546688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.250986004, 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016423218, 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:25.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2756 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:25.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243639+0000", 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.973e-06 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243644+0000", 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3249999999999998e-06 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243648+0000", 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1590000000000003e-06 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259929+0000", 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016280735000000001 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.260004+0000", 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4855000000000001e-05 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.260061+0000", 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6468000000000002e-05 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:25.601 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:25.602 INFO:teuthology.orchestra.run.smithi073.stdout:291 punch from 0x21af263 to 0x21be83a, (0xf5d7 bytes) 2024-09-21T14:21:25.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:25.604 INFO:teuthology.orchestra.run.smithi073.stdout:292 write 0x7056b20 thru 0x7062258 (0xb739 bytes) 2024-09-21T14:21:25.604 INFO:teuthology.orchestra.run.smithi073.stdout:294 read 0x464777d thru 0x464bff0 (0x4874 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:295 read 0x4810270 thru 0x481518d (0x4f1e bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:296 write 0x1e6f611 thru 0x1e74dc0 (0x57b0 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:298 writesame 0x2022151 thru 0x2029df9 data_size 0x2f(0x7ca9 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:299 read 0xc0699c thru 0xc15ff3 (0xf658 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:300 trunc from 0x7062259 to 0x8a3de3 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:302 punch from 0x2a5f7d to 0x2aa8db, (0x495e bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:303 write 0xc0e8775 thru 0xc0effd7 (0x7863 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:304 write 0xcdb022a thru 0xcdbb0ea (0xaec1 bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:306 read 0xa9aeec0 thru 0xa9bad29 (0xbe6a bytes) 2024-09-21T14:21:25.605 INFO:teuthology.orchestra.run.smithi073.stdout:307 trunc from 0xcdbb0eb to 0x8587fc5 2024-09-21T14:21:25.605 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:25.602+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:25.607 INFO:teuthology.orchestra.run.smithi073.stdout:310 trunc from 0x8587fc5 to 0x9bffc1e 2024-09-21T14:21:25.612 INFO:teuthology.orchestra.run.smithi073.stdout:311 write 0x5fb8738 thru 0x5fc12e3 (0x8bac bytes) 2024-09-21T14:21:25.621 INFO:teuthology.orchestra.run.smithi073.stdout:312 write 0x3feb03f thru 0x3ff3b39 (0x8afb bytes) 2024-09-21T14:21:25.626 INFO:teuthology.orchestra.run.smithi073.stdout:314 write 0x42a980f thru 0x42b783e (0xe030 bytes) 2024-09-21T14:21:25.630 INFO:teuthology.orchestra.run.smithi073.stdout:315 write 0x94c5f1d thru 0x94c6eb5 (0xf99 bytes) 2024-09-21T14:21:25.633 INFO:teuthology.orchestra.run.smithi073.stdout:316 writesame 0x3142cd8 thru 0x314692f data_size 0x4(0x3c58 bytes) 2024-09-21T14:21:25.637 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.637 INFO:teuthology.orchestra.run.smithi073.stdout:319 read 0x4de3013 thru 0x4deba22 (0x8a10 bytes) 2024-09-21T14:21:25.648 INFO:teuthology.orchestra.run.smithi073.stdout:320 writesame 0xe1bc660 thru 0xe1cb55b data_size 0xbe(0xeefc bytes) 2024-09-21T14:21:25.648 INFO:teuthology.orchestra.run.smithi073.stdout:322 read 0x78915ce thru 0x78a0b8a (0xf5bd bytes) 2024-09-21T14:21:25.648 INFO:teuthology.orchestra.run.smithi073.stdout:323 punch from 0xbb9e871 to 0xbba4445, (0x5bd4 bytes) 2024-09-21T14:21:25.648 INFO:teuthology.orchestra.run.smithi073.stdout:324 write 0x6fd5ed8 thru 0x6fddca0 (0x7dc9 bytes) 2024-09-21T14:21:25.655 INFO:teuthology.orchestra.run.smithi073.stdout:325 writesame 0x34527b9 thru 0x345befe data_size 0x22(0x9746 bytes) 2024-09-21T14:21:25.663 INFO:teuthology.orchestra.run.smithi073.stdout:326 write 0x1331c75 thru 0x1335b94 (0x3f20 bytes) 2024-09-21T14:21:25.667 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.679 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:25.682 INFO:teuthology.orchestra.run.smithi073.stdout:328 trunc from 0xe1cb55c to 0x1ecc206 2024-09-21T14:21:25.703 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:25.702+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:25.713 INFO:teuthology.orchestra.run.smithi073.stdout:329 write 0xb35459d thru 0xb35f542 (0xafa6 bytes) 2024-09-21T14:21:25.713 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:25.713 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:25.713 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:25.714 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:25.718 INFO:teuthology.orchestra.run.smithi073.stdout:330 write 0xa59398 thru 0xa6351b (0xa184 bytes) 2024-09-21T14:21:25.722 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:25.726 INFO:teuthology.orchestra.run.smithi073.stdout:331 punch from 0x52f419c to 0x52fb878, (0x76dc bytes) 2024-09-21T14:21:25.726 INFO:teuthology.orchestra.run.smithi073.stdout:332 read 0x826f5e3 thru 0x827ab1d (0xb53b bytes) 2024-09-21T14:21:25.727 INFO:teuthology.orchestra.run.smithi073.stdout:334 trunc from 0xb35f543 to 0x9ba3de4 2024-09-21T14:21:25.745 INFO:teuthology.orchestra.run.smithi073.stdout:336 read 0x5e36e04 thru 0x5e42484 (0xb681 bytes) 2024-09-21T14:21:25.745 INFO:teuthology.orchestra.run.smithi073.stdout:337 read 0x91eab3d thru 0x91f2ea9 (0x836d bytes) 2024-09-21T14:21:25.745 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.745 INFO:teuthology.orchestra.run.smithi073.stdout:340 write 0x1d4d57d thru 0x1d5bca0 (0xe724 bytes) 2024-09-21T14:21:25.750 INFO:teuthology.orchestra.run.smithi073.stdout:341 write 0xf8d5db thru 0xf975ba (0x9fe0 bytes) 2024-09-21T14:21:25.757 INFO:teuthology.orchestra.run.smithi073.stdout:342 read 0x40cd60 thru 0x410b14 (0x3db5 bytes) 2024-09-21T14:21:25.757 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.757 INFO:teuthology.orchestra.run.smithi073.stdout:344 write 0x8fb12f2 thru 0x8fb248e (0x119d bytes) 2024-09-21T14:21:25.760 INFO:teuthology.orchestra.run.smithi073.stdout:345 read 0x90c082f thru 0x90c1e5b (0x162d bytes) 2024-09-21T14:21:25.767 INFO:teuthology.orchestra.run.smithi073.stdout:346 write 0xb7655ac thru 0xb76ec95 (0x96ea bytes) 2024-09-21T14:21:25.779 INFO:teuthology.orchestra.run.smithi073.stdout:347 write 0x2b05c08 thru 0x2b096dd (0x3ad6 bytes) 2024-09-21T14:21:25.784 INFO:teuthology.orchestra.run.smithi073.stdout:348 writesame 0x3e322f3 thru 0x3e419d9 data_size 0x9(0xf6e7 bytes) 2024-09-21T14:21:25.791 INFO:teuthology.orchestra.run.smithi073.stdout:349 read 0x1742247 thru 0x1744c8c (0x2a46 bytes) 2024-09-21T14:21:25.791 INFO:teuthology.orchestra.run.smithi073.stdout:350 write 0x60ea129 thru 0x60ebab5 (0x198d bytes) 2024-09-21T14:21:25.803 INFO:teuthology.orchestra.run.smithi073.stdout:351 trunc from 0xb76ec96 to 0x39dafdb 2024-09-21T14:21:25.807 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:25.806+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:25.809 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:25.809 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:25.816 INFO:teuthology.orchestra.run.smithi073.stdout:353 read 0x37e8a53 thru 0x37f5594 (0xcb42 bytes) 2024-09-21T14:21:25.816 INFO:teuthology.orchestra.run.smithi073.stdout:354 writesame 0xe1d160 thru 0xe20632 data_size 0x1(0x34d3 bytes) 2024-09-21T14:21:25.822 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T14:21:24.421166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95445985,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":978,"num_read_kb":1172,"num_write":314,"num_write_kb":3395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218220,"kb_used_data":2040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756500,"statfs":{"total":773094113280,"available":772870656000,"internally_reserved":0,"allocated":2088960,"data_stored":1925383,"data_compressed":66423,"data_compressed_allocated":94208,"data_compressed_original":1179648,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000562"},"pg_stats":[{"pgid":"3.1f","version":"22'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034921+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:22.034921+0000","last_peered":"2024-09-21T14:21:22.034921+0000","last_clean":"2024-09-21T14:21:22.034921+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:22.034921+0000","last_undegraded":"2024-09-21T14:21:22.034921+0000","last_fullsized":"2024-09-21T14:21:22.034921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"23'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.294461+0000","last_change":"2024-09-21T14:21:20.149718+0000","last_active":"2024-09-21T14:21:22.294461+0000","last_peered":"2024-09-21T14:21:22.294461+0000","last_clean":"2024-09-21T14:21:22.294461+0000","last_became_active":"2024-09-21T14:21:18.152993+0000","last_became_peered":"2024-09-21T14:21:18.152993+0000","last_unstale":"2024-09-21T14:21:22.294461+0000","last_undegraded":"2024-09-21T14:21:22.294461+0000","last_fullsized":"2024-09-21T14:21:22.294461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:18.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":3207094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":31,"num_write":1,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152252+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:21.152252+0000","last_peered":"2024-09-21T14:21:21.152252+0000","last_clean":"2024-09-21T14:21:21.152252+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:21.152252+0000","last_undegraded":"2024-09-21T14:21:21.152252+0000","last_fullsized":"2024-09-21T14:21:21.152252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"24'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.313966+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:22.313966+0000","last_peered":"2024-09-21T14:21:22.313966+0000","last_clean":"2024-09-21T14:21:22.313966+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:22.313966+0000","last_undegraded":"2024-09-21T14:21:22.313966+0000","last_fullsized":"2024-09-21T14:21:22.313966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":3995317,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":18,"num_write":3,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975561+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:21.975561+0000","last_peered":"2024-09-21T14:21:21.975561+0000","last_clean":"2024-09-21T14:21:21.975561+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:21.975561+0000","last_undegraded":"2024-09-21T14:21:21.975561+0000","last_fullsized":"2024-09-21T14:21:21.975561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"22'2","reported_seq":47,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.382719+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:22.382719+0000","last_peered":"2024-09-21T14:21:22.382719+0000","last_clean":"2024-09-21T14:21:22.382719+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:22.382719+0000","last_undegraded":"2024-09-21T14:21:22.382719+0000","last_fullsized":"2024-09-21T14:21:22.382719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"24'4","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418472+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:22.418472+0000","last_peered":"2024-09-21T14:21:22.418472+0000","last_clean":"2024-09-21T14:21:22.418472+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:22.418472+0000","last_undegraded":"2024-09-21T14:21:22.418472+0000","last_fullsized":"2024-09-21T14:21:22.418472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":7534650,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":1,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.409252+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:22.409252+0000","last_peered":"2024-09-21T14:21:22.409252+0000","last_clean":"2024-09-21T14:21:22.409252+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:22.409252+0000","last_undegraded":"2024-09-21T14:21:22.409252+0000","last_fullsized":"2024-09-21T14:21:22.409252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":13184888,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":1,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034956+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:22.034956+0000","last_peered":"2024-09-21T14:21:22.034956+0000","last_clean":"2024-09-21T14:21:22.034956+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:22.034956+0000","last_undegraded":"2024-09-21T14:21:22.034956+0000","last_fullsized":"2024-09-21T14:21:22.034956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151529+0000","last_change":"2024-09-21T14:21:20.755882+0000","last_active":"2024-09-21T14:21:21.151529+0000","last_peered":"2024-09-21T14:21:21.151529+0000","last_clean":"2024-09-21T14:21:21.151529+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.151529+0000","last_undegraded":"2024-09-21T14:21:21.151529+0000","last_fullsized":"2024-09-21T14:21:21.151529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:17.531857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.256146+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:21.256146+0000","last_peered":"2024-09-21T14:21:21.256146+0000","last_clean":"2024-09-21T14:21:21.256146+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:21.256146+0000","last_undegraded":"2024-09-21T14:21:21.256146+0000","last_fullsized":"2024-09-21T14:21:21.256146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152175+0000","last_change":"2024-09-21T14:21:20.161641+0000","last_active":"2024-09-21T14:21:21.152175+0000","last_peered":"2024-09-21T14:21:21.152175+0000","last_clean":"2024-09-21T14:21:21.152175+0000","last_became_active":"2024-09-21T14:21:18.152393+0000","last_became_peered":"2024-09-21T14:21:18.152393+0000","last_unstale":"2024-09-21T14:21:21.152175+0000","last_undegraded":"2024-09-21T14:21:21.152175+0000","last_fullsized":"2024-09-21T14:21:21.152175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:55.413999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.877129+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:21.877129+0000","last_peered":"2024-09-21T14:21:21.877129+0000","last_clean":"2024-09-21T14:21:21.877129+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:21.877129+0000","last_undegraded":"2024-09-21T14:21:21.877129+0000","last_fullsized":"2024-09-21T14:21:21.877129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'16","reported_seq":96,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.424523+0000","last_change":"2024-09-21T14:21:20.755817+0000","last_active":"2024-09-21T14:21:22.424523+0000","last_peered":"2024-09-21T14:21:22.424523+0000","last_clean":"2024-09-21T14:21:22.424523+0000","last_became_active":"2024-09-21T14:21:18.150612+0000","last_became_peered":"2024-09-21T14:21:18.150612+0000","last_unstale":"2024-09-21T14:21:22.424523+0000","last_undegraded":"2024-09-21T14:21:22.424523+0000","last_fullsized":"2024-09-21T14:21:22.424523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:50.934485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":180,"num_read_kb":154,"num_write":15,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152523+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:21.152523+0000","last_peered":"2024-09-21T14:21:21.152523+0000","last_clean":"2024-09-21T14:21:21.152523+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:21.152523+0000","last_undegraded":"2024-09-21T14:21:21.152523+0000","last_fullsized":"2024-09-21T14:21:21.152523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.927326+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:21.927326+0000","last_peered":"2024-09-21T14:21:21.927326+0000","last_clean":"2024-09-21T14:21:21.927326+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:21.927326+0000","last_undegraded":"2024-09-21T14:21:21.927326+0000","last_fullsized":"2024-09-21T14:21:21.927326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.151554+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:21.151554+0000","last_peered":"2024-09-21T14:21:21.151554+0000","last_clean":"2024-09-21T14:21:21.151554+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:21.151554+0000","last_undegraded":"2024-09-21T14:21:21.151554+0000","last_fullsized":"2024-09-21T14:21:21.151554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875923+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:21.875923+0000","last_peered":"2024-09-21T14:21:21.875923+0000","last_clean":"2024-09-21T14:21:21.875923+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:21.875923+0000","last_undegraded":"2024-09-21T14:21:21.875923+0000","last_fullsized":"2024-09-21T14:21:21.875923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.947981+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:21.947981+0000","last_peered":"2024-09-21T14:21:21.947981+0000","last_clean":"2024-09-21T14:21:21.947981+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:21.947981+0000","last_undegraded":"2024-09-21T14:21:21.947981+0000","last_fullsized":"2024-09-21T14:21:21.947981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'6","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.416170+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:22.416170+0000","last_peered":"2024-09-21T14:21:22.416170+0000","last_clean":"2024-09-21T14:21:22.416170+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:22.416170+0000","last_undegraded":"2024-09-21T14:21:22.416170+0000","last_fullsized":"2024-09-21T14:21:22.416170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":15209098,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":125,"num_write":3,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.975446+0000","last_change":"2024-09-21T14:21:20.755906+0000","last_active":"2024-09-21T14:21:21.975446+0000","last_peered":"2024-09-21T14:21:21.975446+0000","last_clean":"2024-09-21T14:21:21.975446+0000","last_became_active":"2024-09-21T14:21:18.151157+0000","last_became_peered":"2024-09-21T14:21:18.151157+0000","last_unstale":"2024-09-21T14:21:21.975446+0000","last_undegraded":"2024-09-21T14:21:21.975446+0000","last_fullsized":"2024-09-21T14:21:21.975446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:25.967224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152075+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:21.152075+0000","last_peered":"2024-09-21T14:21:21.152075+0000","last_clean":"2024-09-21T14:21:21.152075+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:21.152075+0000","last_undegraded":"2024-09-21T14:21:21.152075+0000","last_fullsized":"2024-09-21T14:21:21.152075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875956+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:21.875956+0000","last_peered":"2024-09-21T14:21:21.875956+0000","last_clean":"2024-09-21T14:21:21.875956+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:21.875956+0000","last_undegraded":"2024-09-21T14:21:21.875956+0000","last_fullsized":"2024-09-21T14:21:21.875956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.034849+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:22.034849+0000","last_peered":"2024-09-21T14:21:22.034849+0000","last_clean":"2024-09-21T14:21:22.034849+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:22.034849+0000","last_undegraded":"2024-09-21T14:21:22.034849+0000","last_fullsized":"2024-09-21T14:21:22.034849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.255796+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:21.255796+0000","last_peered":"2024-09-21T14:21:21.255796+0000","last_clean":"2024-09-21T14:21:21.255796+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:21.255796+0000","last_undegraded":"2024-09-21T14:21:21.255796+0000","last_fullsized":"2024-09-21T14:21:21.255796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"24'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.182951+0000","last_change":"2024-09-21T14:21:20.151933+0000","last_active":"2024-09-21T14:21:22.182951+0000","last_peered":"2024-09-21T14:21:22.182951+0000","last_clean":"2024-09-21T14:21:22.182951+0000","last_became_active":"2024-09-21T14:21:18.147902+0000","last_became_peered":"2024-09-21T14:21:18.147902+0000","last_unstale":"2024-09-21T14:21:22.182951+0000","last_undegraded":"2024-09-21T14:21:22.182951+0000","last_fullsized":"2024-09-21T14:21:22.182951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:09.190450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":41,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152478+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:21.152478+0000","last_peered":"2024-09-21T14:21:21.152478+0000","last_clean":"2024-09-21T14:21:21.152478+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:21.152478+0000","last_undegraded":"2024-09-21T14:21:21.152478+0000","last_fullsized":"2024-09-21T14:21:21.152478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"24'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.355019+0000","last_change":"2024-09-21T14:21:20.145954+0000","last_active":"2024-09-21T14:21:22.355019+0000","last_peered":"2024-09-21T14:21:22.355019+0000","last_clean":"2024-09-21T14:21:22.355019+0000","last_became_active":"2024-09-21T14:21:18.152479+0000","last_became_peered":"2024-09-21T14:21:18.152479+0000","last_unstale":"2024-09-21T14:21:22.355019+0000","last_undegraded":"2024-09-21T14:21:22.355019+0000","last_fullsized":"2024-09-21T14:21:22.355019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:27.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":16777232,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":6,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.875848+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:21.875848+0000","last_peered":"2024-09-21T14:21:21.875848+0000","last_clean":"2024-09-21T14:21:21.875848+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:21.875848+0000","last_undegraded":"2024-09-21T14:21:21.875848+0000","last_fullsized":"2024-09-21T14:21:21.875848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"24'37","reported_seq":59,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.418646+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:22.418646+0000","last_peered":"2024-09-21T14:21:22.418646+0000","last_clean":"2024-09-21T14:21:22.418646+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:22.418646+0000","last_undegraded":"2024-09-21T14:21:22.418646+0000","last_fullsized":"2024-09-21T14:21:22.418646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":6837908,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":194,"num_write":53,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"24'17","reported_seq":35,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.379636+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:22.379636+0000","last_peered":"2024-09-21T14:21:22.379636+0000","last_clean":"2024-09-21T14:21:22.379636+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:22.379636+0000","last_undegraded":"2024-09-21T14:21:22.379636+0000","last_fullsized":"2024-09-21T14:21:22.379636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":36,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":51,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"24'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.354567+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:22.354567+0000","last_peered":"2024-09-21T14:21:22.354567+0000","last_clean":"2024-09-21T14:21:22.354567+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:22.354567+0000","last_undegraded":"2024-09-21T14:21:22.354567+0000","last_fullsized":"2024-09-21T14:21:22.354567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":16167918,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":35,"num_write":3,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"23'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.345351+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:22.345351+0000","last_peered":"2024-09-21T14:21:22.345351+0000","last_clean":"2024-09-21T14:21:22.345351+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:22.345351+0000","last_undegraded":"2024-09-21T14:21:22.345351+0000","last_fullsized":"2024-09-21T14:21:22.345351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":1593100,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152309+0000","last_change":"2024-09-21T14:21:20.162015+0000","last_active":"2024-09-21T14:21:21.152309+0000","last_peered":"2024-09-21T14:21:21.152309+0000","last_clean":"2024-09-21T14:21:21.152309+0000","last_became_active":"2024-09-21T14:21:18.143760+0000","last_became_peered":"2024-09-21T14:21:18.143760+0000","last_unstale":"2024-09-21T14:21:21.152309+0000","last_undegraded":"2024-09-21T14:21:21.152309+0000","last_fullsized":"2024-09-21T14:21:21.152309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.757040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"23'4","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.015396+0000","last_change":"2024-09-21T14:21:20.756021+0000","last_active":"2024-09-21T14:21:22.015396+0000","last_peered":"2024-09-21T14:21:22.015396+0000","last_clean":"2024-09-21T14:21:22.015396+0000","last_became_active":"2024-09-21T14:21:18.152859+0000","last_became_peered":"2024-09-21T14:21:18.152859+0000","last_unstale":"2024-09-21T14:21:22.015396+0000","last_undegraded":"2024-09-21T14:21:22.015396+0000","last_fullsized":"2024-09-21T14:21:22.015396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.438917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"24'21","reported_seq":163,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.285127+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:22.285127+0000","last_peered":"2024-09-21T14:21:22.285127+0000","last_clean":"2024-09-21T14:21:22.285127+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:22.285127+0000","last_undegraded":"2024-09-21T14:21:22.285127+0000","last_fullsized":"2024-09-21T14:21:22.285127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":359,"num_read_kb":311,"num_write":19,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.876974+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:21.876974+0000","last_peered":"2024-09-21T14:21:21.876974+0000","last_clean":"2024-09-21T14:21:21.876974+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:21.876974+0000","last_undegraded":"2024-09-21T14:21:21.876974+0000","last_fullsized":"2024-09-21T14:21:21.876974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:21:22.426849+0000","last_change":"2024-09-21T14:21:20.160536+0000","last_active":"2024-09-21T14:21:22.426849+0000","last_peered":"2024-09-21T14:21:22.426849+0000","last_clean":"2024-09-21T14:21:22.426849+0000","last_became_active":"2024-09-21T14:21:18.151161+0000","last_became_peered":"2024-09-21T14:21:18.151161+0000","last_unstale":"2024-09-21T14:21:22.426849+0000","last_undegraded":"2024-09-21T14:21:22.426849+0000","last_fullsized":"2024-09-21T14:21:22.426849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:48.604891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":10348281,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":119,"num_write":8,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.958641+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:21.958641+0000","last_peered":"2024-09-21T14:21:21.958641+0000","last_clean":"2024-09-21T14:21:21.958641+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:21.958641+0000","last_undegraded":"2024-09-21T14:21:21.958641+0000","last_fullsized":"2024-09-21T14:21:21.958641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T14:21:21.152151+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:21.152151+0000","last_peered":"2024-09-21T14:21:21.152151+0000","last_clean":"2024-09-21T14:21:21.152151+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:21.152151+0000","last_undegraded":"2024-09-21T14:21:21.152151+0000","last_fullsized":"2024-09-21T14:21:21.152151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":94855598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":862,"num_read_kb":1072,"num_write":153,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":527422,"data_compressed":18878,"data_compressed_allocated":49152,"data_compressed_original":385024,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":83424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":92810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":672841,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":83473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":679631,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":89233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":134719,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":89252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":61208,"data_compressed":2228,"data_compressed_allocated":4096,"data_compressed_original":45056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":145636,"data_compressed":6414,"data_compressed_allocated":16384,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":66052,"data_compressed":3021,"data_compressed_allocated":8192,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":162317,"data_compressed":6046,"data_compressed_allocated":16384,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":31721,"data_compressed":1169,"data_compressed_allocated":4096,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":10839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:25.825 INFO:tasks.thrashosds.thrasher:Setting 3.1a to [4, 2, 7, 0] 2024-09-21T14:21:25.825 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1a', '4', '2', '7', '0'] 2024-09-21T14:21:25.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1a 4 2 7 0 2024-09-21T14:21:25.826 INFO:teuthology.orchestra.run.smithi073.stdout:355 punch from 0x59aa0 to 0x65f23, (0xc483 bytes) 2024-09-21T14:21:25.826 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:25.826 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:25.841 INFO:teuthology.orchestra.run.smithi073.stdout:356 writesame 0xdf5d95e thru 0xdf6af88 data_size 0x1(0xd62b bytes) 2024-09-21T14:21:25.841 INFO:teuthology.orchestra.run.smithi073.stdout:357 read 0xb5cc4db thru 0xb5d2301 (0x5e27 bytes) 2024-09-21T14:21:25.842 INFO:teuthology.orchestra.run.smithi073.stdout:358 read 0x2d56cd4 thru 0x2d61ea0 (0xb1cd bytes) 2024-09-21T14:21:25.842 INFO:teuthology.orchestra.run.smithi073.stdout:360 punch from 0xa2e3135 to 0xa2e82c7, (0x5192 bytes) 2024-09-21T14:21:25.842 INFO:teuthology.orchestra.run.smithi073.stdout:361 write 0xc06d8c2 thru 0xc07d293 (0xf9d2 bytes) 2024-09-21T14:21:25.843 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:25.844 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:25.859 INFO:teuthology.orchestra.run.smithi073.stdout:363 trunc from 0xdf6af89 to 0x5ab0970 2024-09-21T14:21:25.862 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:25.862 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:25.870 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:25.870 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:25.870 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:25.870 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:25.870 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:364 read 0x1f46e58 thru 0x1f56055 (0xf1fe bytes) 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:366 read 0x3cfb3b7 thru 0x3cfe2c6 (0x2f10 bytes) 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:368 read 0x2f575fa thru 0x2f67432 (0xfe39 bytes) 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:369 read 0x4a6767d thru 0x4a6fb56 (0x84da bytes) 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:371 punch from 0x2b6dcad to 0x2b7d879, (0xfbcc bytes) 2024-09-21T14:21:25.871 INFO:teuthology.orchestra.run.smithi073.stdout:372 read 0x5313f5 thru 0x53d044 (0xbc50 bytes) 2024-09-21T14:21:25.872 INFO:teuthology.orchestra.run.smithi073.stdout:373 write 0x2e5d405 thru 0x2e6c9f0 (0xf5ec bytes) 2024-09-21T14:21:25.879 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:25.881 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:25.882 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:25.886 INFO:teuthology.orchestra.run.smithi073.stdout:374 writesame 0x96e4822 thru 0x96f21d8 data_size 0x5(0xd9b7 bytes) 2024-09-21T14:21:25.886 INFO:teuthology.orchestra.run.smithi073.stdout:376 punch from 0x3c044fa to 0x3c0dd4c, (0x9852 bytes) 2024-09-21T14:21:25.887 INFO:teuthology.orchestra.run.smithi073.stdout:379 punch from 0x4ecb387 to 0x4ed4b07, (0x9780 bytes) 2024-09-21T14:21:25.887 INFO:teuthology.orchestra.run.smithi073.stdout:380 write 0x944edfc thru 0x94522e4 (0x34e9 bytes) 2024-09-21T14:21:25.894 INFO:teuthology.orchestra.run.smithi073.stdout:381 punch from 0x7ce6832 to 0x7ced32a, (0x6af8 bytes) 2024-09-21T14:21:25.900 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:25.901 INFO:teuthology.orchestra.run.smithi073.stdout:383 trunc from 0x96f21d9 to 0x2e64236 2024-09-21T14:21:25.917 INFO:teuthology.orchestra.run.smithi073.stdout:384 read 0x2a3564d thru 0x2a37f3f (0x28f3 bytes) 2024-09-21T14:21:25.917 INFO:teuthology.orchestra.run.smithi073.stdout:385 punch from 0x2899c9c to 0x28a8c6f, (0xefd3 bytes) 2024-09-21T14:21:25.919 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:25.919 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:25.920 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:25.919+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:386 writesame 0xafacca2 thru 0xafb9adf data_size 0x1(0xce3e bytes) 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:389 read 0x778fc33 thru 0x7798778 (0x8b46 bytes) 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:390 punch from 0x7111481 to 0x7118e66, (0x79e5 bytes) 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:392 read 0x390ef1f thru 0x3915b5c (0x6c3e bytes) 2024-09-21T14:21:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:393 write 0x2abb26f thru 0x2abd8eb (0x267d bytes) 2024-09-21T14:21:25.936 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:25.937 INFO:teuthology.orchestra.run.smithi073.stdout:394 read 0x24d81e3 thru 0x24dbefc (0x3d1a bytes) 2024-09-21T14:21:25.937 INFO:teuthology.orchestra.run.smithi073.stdout:395 punch from 0x35663b2 to 0x356ca62, (0x66b0 bytes) 2024-09-21T14:21:25.937 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.937 INFO:teuthology.orchestra.run.smithi073.stdout:397 write 0xcdb5f4 thru 0xce509b (0x9aa8 bytes) 2024-09-21T14:21:25.943 INFO:teuthology.orchestra.run.smithi073.stdout:398 read 0x3d7050 thru 0x3e1e7a (0xae2b bytes) 2024-09-21T14:21:25.944 INFO:teuthology.orchestra.run.smithi073.stdout:399 writesame 0x43ae81f thru 0x43bd450 data_size 0x31(0xec32 bytes) 2024-09-21T14:21:25.949 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:25.949 INFO:teuthology.orchestra.run.smithi073.stdout:401 write 0x3765e0 thru 0x37cf23 (0x6944 bytes) 2024-09-21T14:21:25.960 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:25.960 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:25.971 INFO:teuthology.orchestra.run.smithi073.stdout:402 writesame 0xce06a83 thru 0xce10857 data_size 0x1(0x9dd5 bytes) 2024-09-21T14:21:25.971 INFO:teuthology.orchestra.run.smithi073.stdout:403 read 0x9068d7e thru 0x90708f0 (0x7b73 bytes) 2024-09-21T14:21:25.971 INFO:teuthology.orchestra.run.smithi073.stdout:404 write 0xa6e74a7 thru 0xa6eec3e (0x7798 bytes) 2024-09-21T14:21:25.977 INFO:teuthology.orchestra.run.smithi073.stdout:405 write 0x4ebb81f thru 0x4ec3f28 (0x870a bytes) 2024-09-21T14:21:25.986 INFO:teuthology.orchestra.run.smithi073.stdout:406 punch from 0x68f6688 to 0x68f9467, (0x2ddf bytes) 2024-09-21T14:21:25.992 INFO:teuthology.orchestra.run.smithi073.stdout:407 writesame 0xd8d1792 thru 0xd8d1ab7 data_size 0xd(0x326 bytes) 2024-09-21T14:21:25.993 INFO:teuthology.orchestra.run.smithi073.stdout:409 write 0xbe101cb thru 0xbe1ebbc (0xe9f2 bytes) 2024-09-21T14:21:25.999 INFO:teuthology.orchestra.run.smithi073.stdout:410 read 0x76f5294 thru 0x76f8658 (0x33c5 bytes) 2024-09-21T14:21:25.999 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.005 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:26.004+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:26.012 INFO:teuthology.orchestra.run.smithi073.stdout:412 trunc from 0xd8d1ab8 to 0x299d340 2024-09-21T14:21:26.030 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.030 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.038 INFO:teuthology.orchestra.run.smithi073.stdout:415 write 0x6ca39e0 thru 0x6ca3b2d (0x14e bytes) 2024-09-21T14:21:26.039 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:26.041 INFO:teuthology.orchestra.run.smithi073.stdout:417 read 0x3f485d5 thru 0x3f52f1f (0xa94b bytes) 2024-09-21T14:21:26.055 INFO:teuthology.orchestra.run.smithi073.stdout:419 write 0xe4c5b07 thru 0xe4c7967 (0x1e61 bytes) 2024-09-21T14:21:26.058 INFO:teuthology.orchestra.run.smithi073.stdout:420 write 0xbfa450 thru 0xbfd3bc (0x2f6d bytes) 2024-09-21T14:21:26.062 INFO:teuthology.orchestra.run.smithi073.stdout:421 read 0xb851568 thru 0xb85ceaf (0xb948 bytes) 2024-09-21T14:21:26.077 INFO:teuthology.orchestra.run.smithi073.stdout:422 trunc from 0xe4c7968 to 0x1a3aad5 2024-09-21T14:21:26.090 INFO:teuthology.orchestra.run.smithi073.stdout:425 trunc from 0x1a3aad5 to 0x4274ffe 2024-09-21T14:21:26.114 INFO:teuthology.orchestra.run.smithi073.stdout:426 write 0xe9dd572 thru 0xe9ec3b9 (0xee48 bytes) 2024-09-21T14:21:26.120 INFO:teuthology.orchestra.run.smithi073.stdout:427 punch from 0x58558ec to 0x586306b, (0xd77f bytes) 2024-09-21T14:21:26.120 INFO:teuthology.orchestra.run.smithi073.stdout:428 writesame 0x339b2ea thru 0x339e6a5 data_size 0x2c(0x33bc bytes) 2024-09-21T14:21:26.123 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:26.123+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:26.125 INFO:teuthology.orchestra.run.smithi073.stdout:429 read 0xf7ebb7 thru 0xf8cc0a (0xe054 bytes) 2024-09-21T14:21:26.126 INFO:teuthology.orchestra.run.smithi073.stdout:432 read 0xa078e78 thru 0xa07e6bf (0x5848 bytes) 2024-09-21T14:21:26.126 INFO:teuthology.orchestra.run.smithi073.stdout:434 write 0x1c53904 thru 0x1c5cb3c (0x9239 bytes) 2024-09-21T14:21:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:21:26.139 INFO:teuthology.orchestra.run.smithi073.stdout:435 write 0xb871370 thru 0xb8719d8 (0x669 bytes) 2024-09-21T14:21:26.150 INFO:teuthology.orchestra.run.smithi073.stdout:436 read 0x797c6e1 thru 0x798c2d2 (0xfbf2 bytes) 2024-09-21T14:21:26.150 INFO:teuthology.orchestra.run.smithi073.stdout:438 read 0x5fc2327 thru 0x5fc72bb (0x4f95 bytes) 2024-09-21T14:21:26.150 INFO:teuthology.orchestra.run.smithi073.stdout:440 read 0x9fcc9be thru 0x9fdb3f1 (0xea34 bytes) 2024-09-21T14:21:26.156 INFO:teuthology.orchestra.run.smithi073.stdout:441 write 0xed8e28c thru 0xed93379 (0x50ee bytes) 2024-09-21T14:21:26.161 INFO:teuthology.orchestra.run.smithi073.stdout:442 read 0xc769303 thru 0xc7771a5 (0xdea3 bytes) 2024-09-21T14:21:26.161 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.161 INFO:teuthology.orchestra.run.smithi073.stdout:444 write 0xddb8876 thru 0xddbc73c (0x3ec7 bytes) 2024-09-21T14:21:26.166 INFO:teuthology.orchestra.run.smithi073.stdout:446 read 0x22a0141 thru 0x22a366e (0x352e bytes) 2024-09-21T14:21:26.167 INFO:teuthology.orchestra.run.smithi073.stdout:447 read 0xb0dc9cc thru 0xb0e9318 (0xc94d bytes) 2024-09-21T14:21:26.167 INFO:teuthology.orchestra.run.smithi073.stdout:448 write 0x23523e4 thru 0x235ff8c (0xdba9 bytes) 2024-09-21T14:21:26.174 INFO:teuthology.orchestra.run.smithi073.stdout:449 write 0xbb9e96e thru 0xbbabc59 (0xd2ec bytes) 2024-09-21T14:21:26.182 INFO:teuthology.orchestra.run.smithi073.stdout:450 write 0xf6eda4 thru 0xf732bc (0x4519 bytes) 2024-09-21T14:21:26.184 INFO:teuthology.orchestra.run.smithi073.stdout:451 read 0x28c999a thru 0x28d4ad5 (0xb13c bytes) 2024-09-21T14:21:26.184 INFO:teuthology.orchestra.run.smithi073.stdout:454 read 0xdc8a536 thru 0xdc8cd6a (0x2835 bytes) 2024-09-21T14:21:26.185 INFO:teuthology.orchestra.run.smithi073.stdout:455 write 0x635d751 thru 0x635ff6e (0x281e bytes) 2024-09-21T14:21:26.187 INFO:teuthology.orchestra.run.smithi073.stdout:456 write 0x9f14031 thru 0x9f18c52 (0x4c22 bytes) 2024-09-21T14:21:26.189 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.189 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:26.206 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:26.206+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:26.207 INFO:teuthology.orchestra.run.smithi073.stdout:457 trunc from 0xed9337a to 0x1375451 2024-09-21T14:21:26.227 INFO:teuthology.orchestra.run.smithi073.stdout:458 punch from 0x55b55e to 0x565cdc, (0xa77e bytes) 2024-09-21T14:21:26.234 INFO:teuthology.orchestra.run.smithi073.stdout:459 trunc from 0x1375451 to 0x7287090 2024-09-21T14:21:26.242 INFO:teuthology.orchestra.run.smithi073.stdout:460 punch from 0x71291e to 0x71e3b7, (0xba99 bytes) 2024-09-21T14:21:26.242 INFO:teuthology.orchestra.run.smithi073.stdout:461 read 0x3063823 thru 0x3072611 (0xedef bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:464 writesame 0x8764bea thru 0x8772a3c data_size 0x1(0xde53 bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:466 punch from 0x30ff837 to 0x31016ae, (0x1e77 bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:467 read 0x431a99f thru 0x431e910 (0x3f72 bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:470 read 0x1eef725 thru 0x1efb921 (0xc1fd bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:471 punch from 0x7ce081 to 0x7d3644, (0x55c3 bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:472 read 0x6cdc876 thru 0x6ce60e9 (0x9874 bytes) 2024-09-21T14:21:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:473 read 0x855fb7f thru 0x85633ab (0x382d bytes) 2024-09-21T14:21:26.250 INFO:teuthology.orchestra.run.smithi073.stdout:474 write 0x3f0334a thru 0x3f08e98 (0x5b4f bytes) 2024-09-21T14:21:26.254 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.254 INFO:teuthology.orchestra.run.smithi073.stdout:479 read 0x695247 thru 0x698595 (0x334f bytes) 2024-09-21T14:21:26.254 INFO:teuthology.orchestra.run.smithi073.stdout:480 write 0x69d0184 thru 0x69d2644 (0x24c1 bytes) 2024-09-21T14:21:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:481 write 0x9ca5b68 thru 0x9cb3ba6 (0xe03f bytes) 2024-09-21T14:21:26.279 INFO:teuthology.orchestra.run.smithi073.stdout:482 write 0xcac59f2 thru 0xcac9b73 (0x4182 bytes) 2024-09-21T14:21:26.285 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.285 INFO:teuthology.orchestra.run.smithi073.stdout:485 write 0x1b21181 thru 0x1b262ce (0x514e bytes) 2024-09-21T14:21:26.292 INFO:teuthology.orchestra.run.smithi073.stdout:486 read 0x69f3add thru 0x69febc9 (0xb0ed bytes) 2024-09-21T14:21:26.292 INFO:teuthology.orchestra.run.smithi073.stdout:487 writesame 0xbe3b37e thru 0xbe41ad6 data_size 0x1(0x6759 bytes) 2024-09-21T14:21:26.305 INFO:teuthology.orchestra.run.smithi073.stdout:488 write 0x19bedc5 thru 0x19ca9a9 (0xbbe5 bytes) 2024-09-21T14:21:26.307 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:26.308+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:26.317 INFO:teuthology.orchestra.run.smithi073.stdout:489 write 0xb4b80fe thru 0xb4c75ac (0xf4af bytes) 2024-09-21T14:21:26.323 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.323 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.323 INFO:teuthology.orchestra.run.smithi073.stdout:493 trunc from 0xcac9b74 to 0xc0be419 2024-09-21T14:21:26.332 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.338 INFO:teuthology.orchestra.run.smithi073.stdout:496 write 0xc7cc105 thru 0xc7d50c5 (0x8fc1 bytes) 2024-09-21T14:21:26.342 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.342 INFO:teuthology.orchestra.run.smithi073.stdout:499 write 0xb8c12b3 thru 0xb8ce509 (0xd257 bytes) 2024-09-21T14:21:26.347 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.347 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.349 INFO:teuthology.orchestra.run.smithi073.stdout:500 punch from 0x99d999f to 0x99dc030, (0x2691 bytes) 2024-09-21T14:21:26.349 INFO:teuthology.orchestra.run.smithi073.stdout:501 read 0x6e0952d thru 0x6e0d6c9 (0x419d bytes) 2024-09-21T14:21:26.349 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.349 INFO:teuthology.orchestra.run.smithi073.stdout:503 read 0x4070d74 thru 0x407da54 (0xcce1 bytes) 2024-09-21T14:21:26.357 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:26.358 INFO:teuthology.orchestra.run.smithi073.stdout:504 trunc from 0xc7d50c6 to 0x58a7b14 2024-09-21T14:21:26.375 INFO:teuthology.orchestra.run.smithi073.stdout:506 trunc from 0x58a7b14 to 0x5486bd2 2024-09-21T14:21:26.396 INFO:teuthology.orchestra.run.smithi073.stdout:508 writesame 0xe1326f2 thru 0xe140548 data_size 0x1(0xde57 bytes) 2024-09-21T14:21:26.397 INFO:teuthology.orchestra.run.smithi073.stdout:509 punch from 0xde6827a to 0xde70c6d, (0x89f3 bytes) 2024-09-21T14:21:26.397 INFO:teuthology.orchestra.run.smithi073.stdout:510 write 0x6a97d40 thru 0x6a99d6c (0x202d bytes) 2024-09-21T14:21:26.400 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.400 INFO:teuthology.orchestra.run.smithi073.stdout:512 write 0xa8145fa thru 0xa823490 (0xee97 bytes) 2024-09-21T14:21:26.404 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.405 INFO:teuthology.orchestra.run.smithi073.stdout:514 writesame 0x42d9be2 thru 0x42e07a9 data_size 0x4(0x6bc8 bytes) 2024-09-21T14:21:26.408 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:26.408+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:26.413 INFO:teuthology.orchestra.run.smithi073.stdout:515 write 0xa5eaab1 thru 0xa5f1705 (0x6c55 bytes) 2024-09-21T14:21:26.419 INFO:teuthology.orchestra.run.smithi073.stdout:516 writesame 0x73cd979 thru 0x73dc4d9 data_size 0x1(0xeb61 bytes) 2024-09-21T14:21:26.437 INFO:teuthology.orchestra.run.smithi073.stdout:517 write 0xcf48265 thru 0xcf4c975 (0x4711 bytes) 2024-09-21T14:21:26.452 INFO:teuthology.orchestra.run.smithi073.stdout:518 trunc from 0xe140549 to 0x711fba5 2024-09-21T14:21:26.464 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.464 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:26.479 INFO:teuthology.orchestra.run.smithi073.stdout:519 trunc from 0x711fba5 to 0x4f8eeb 2024-09-21T14:21:26.504 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.515 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:26.514+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:26.521 INFO:teuthology.orchestra.run.smithi073.stdout:522 writesame 0xa7aadba thru 0xa7abb2a data_size 0x1f(0xd71 bytes) 2024-09-21T14:21:26.524 INFO:teuthology.orchestra.run.smithi073.stdout:523 trunc from 0xa7abb2b to 0x86a77b4 2024-09-21T14:21:26.529 INFO:teuthology.orchestra.run.smithi073.stdout:525 writesame 0xa4b03b thru 0xa5aff3 data_size 0x1(0xffb9 bytes) 2024-09-21T14:21:26.550 INFO:teuthology.orchestra.run.smithi073.stdout:526 punch from 0x1592721 to 0x1592ec6, (0x7a5 bytes) 2024-09-21T14:21:26.551 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.551 INFO:teuthology.orchestra.run.smithi073.stdout:530 write 0x7965bce thru 0x796e6fb (0x8b2e bytes) 2024-09-21T14:21:26.560 INFO:teuthology.orchestra.run.smithi073.stdout:531 trunc from 0x86a77b4 to 0x7e9d8b2 2024-09-21T14:21:26.565 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.565 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xeb7af62 2024-09-21T14:21:26.573 INFO:teuthology.orchestra.run.smithi073.stdout:533 trunc from 0x7e9d8b2 to 0xeb7af62 2024-09-21T14:21:26.578 INFO:teuthology.orchestra.run.smithi073.stdout:535 punch from 0x9e828e to 0x9ef13b, (0x6ead bytes) 2024-09-21T14:21:26.583 INFO:teuthology.orchestra.run.smithi073.stdout:536 trunc from 0xeb7af62 to 0xaa92c98 2024-09-21T14:21:26.589 INFO:teuthology.orchestra.run.smithi073.stdout:537 read 0x76c0e69 thru 0x76ce7e7 (0xd97f bytes) 2024-09-21T14:21:26.589 INFO:teuthology.orchestra.run.smithi073.stdout:538 read 0xdd26e2 thru 0xdde8c1 (0xc1e0 bytes) 2024-09-21T14:21:26.589 INFO:teuthology.orchestra.run.smithi073.stdout:540 read 0x9c0f536 thru 0x9c15165 (0x5c30 bytes) 2024-09-21T14:21:26.589 INFO:teuthology.orchestra.run.smithi073.stdout:541 read 0x10f780 thru 0x11e868 (0xf0e9 bytes) 2024-09-21T14:21:26.590 INFO:teuthology.orchestra.run.smithi073.stdout:542 punch from 0x5cde8e5 to 0x5ce9f2d, (0xb648 bytes) 2024-09-21T14:21:26.590 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.590 INFO:teuthology.orchestra.run.smithi073.stdout:544 write 0x7872ae thru 0x792074 (0xadc7 bytes) 2024-09-21T14:21:26.596 INFO:teuthology.orchestra.run.smithi073.stdout:546 write 0x9829a81 thru 0x982cc8f (0x320f bytes) 2024-09-21T14:21:26.601 INFO:teuthology.orchestra.run.smithi073.stdout:547 punch from 0x46e86d1 to 0x46eb13d, (0x2a6c bytes) 2024-09-21T14:21:26.606 INFO:teuthology.orchestra.run.smithi073.stdout:549 write 0xb15d01f thru 0xb169c1a (0xcbfc bytes) 2024-09-21T14:21:26.610 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:26.609+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:26.612 INFO:teuthology.orchestra.run.smithi073.stdout:550 read 0x1f4c96e thru 0x1f5b6e1 (0xed74 bytes) 2024-09-21T14:21:26.612 INFO:teuthology.orchestra.run.smithi073.stdout:551 write 0x7f7bb6c thru 0x7f82283 (0x6718 bytes) 2024-09-21T14:21:26.617 INFO:teuthology.orchestra.run.smithi073.stdout:552 write 0x7ab8aff thru 0x7abb1d1 (0x26d3 bytes) 2024-09-21T14:21:26.619 INFO:teuthology.orchestra.run.smithi073.stdout:554 write 0x602e7f7 thru 0x6039212 (0xaa1c bytes) 2024-09-21T14:21:26.621 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.621 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.624 INFO:teuthology.orchestra.run.smithi073.stdout:555 writesame 0xa9ec9bd thru 0xa9fb170 data_size 0x6(0xe7b4 bytes) 2024-09-21T14:21:26.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:26.637 INFO:teuthology.orchestra.run.smithi073.stdout:556 trunc from 0xb169c1b to 0x6f3d013 2024-09-21T14:21:26.649 INFO:teuthology.orchestra.run.smithi073.stdout:557 punch from 0x1048e5c to 0x1052f9a, (0xa13e bytes) 2024-09-21T14:21:26.649 INFO:teuthology.orchestra.run.smithi073.stdout:558 write 0x545aae8 thru 0x5469b33 (0xf04c bytes) 2024-09-21T14:21:26.658 INFO:teuthology.orchestra.run.smithi073.stdout:561 write 0x33fc9b3 thru 0x33fff39 (0x3587 bytes) 2024-09-21T14:21:26.666 INFO:teuthology.orchestra.run.smithi073.stdout:562 writesame 0x7485f3d thru 0x7488fc0 data_size 0x3(0x3084 bytes) 2024-09-21T14:21:26.671 INFO:teuthology.orchestra.run.smithi073.stdout:563 trunc from 0x7488fc1 to 0xb135672 2024-09-21T14:21:26.675 INFO:teuthology.orchestra.run.smithi073.stdout:564 read 0x97e8f5 thru 0x982f97 (0x46a3 bytes) 2024-09-21T14:21:26.676 INFO:teuthology.orchestra.run.smithi073.stdout:566 write 0x2caeecd thru 0x2cb2669 (0x379d bytes) 2024-09-21T14:21:26.682 INFO:teuthology.orchestra.run.smithi073.stdout:568 punch from 0x1e2bca7 to 0x1e30d2a, (0x5083 bytes) 2024-09-21T14:21:26.682 INFO:teuthology.orchestra.run.smithi073.stdout:569 write 0x463fd54 thru 0x4646544 (0x67f1 bytes) 2024-09-21T14:21:26.687 INFO:teuthology.orchestra.run.smithi073.stdout:570 read 0x37a787c thru 0x37ac313 (0x4a98 bytes) 2024-09-21T14:21:26.693 INFO:teuthology.orchestra.run.smithi073.stdout:574 write 0xb7d135d thru 0xb7e0490 (0xf134 bytes) 2024-09-21T14:21:26.706 INFO:teuthology.orchestra.run.smithi073.stdout:575 write 0xbcd6b37 thru 0xbcdfd36 (0x9200 bytes) 2024-09-21T14:21:26.711 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:26.711+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:26.712 INFO:teuthology.orchestra.run.smithi073.stdout:576 punch from 0x168b521 to 0x169899c, (0xd47b bytes) 2024-09-21T14:21:26.712 INFO:teuthology.orchestra.run.smithi073.stdout:577 punch from 0x9988f06 to 0x9992405, (0x94ff bytes) 2024-09-21T14:21:26.712 INFO:teuthology.orchestra.run.smithi073.stdout:578 writesame 0xa9003e8 thru 0xa9069d0 data_size 0x1(0x65e9 bytes) 2024-09-21T14:21:26.720 INFO:teuthology.orchestra.run.smithi073.stdout:580 writesame 0x64aa204 thru 0x64acb2b data_size 0xc(0x2928 bytes) 2024-09-21T14:21:26.727 INFO:teuthology.orchestra.run.smithi073.stdout:581 write 0x6c8199d thru 0x6c8e8ca (0xcf2e bytes) 2024-09-21T14:21:26.735 INFO:teuthology.orchestra.run.smithi073.stdout:582 write 0x7e437a0 thru 0x7e4cfff (0x9860 bytes) 2024-09-21T14:21:26.745 INFO:teuthology.orchestra.run.smithi073.stdout:584 write 0x706ddaa thru 0x7078e58 (0xb0af bytes) 2024-09-21T14:21:26.753 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.753 INFO:teuthology.orchestra.run.smithi073.stdout:590 read 0xb0f7acd thru 0xb10146c (0x99a0 bytes) 2024-09-21T14:21:26.753 INFO:teuthology.orchestra.run.smithi073.stdout:593 write 0x77f045c thru 0x77f2c0e (0x27b3 bytes) 2024-09-21T14:21:26.755 INFO:teuthology.orchestra.run.smithi073.stdout:594 writesame 0xa931460 thru 0xa93d988 data_size 0x1(0xc529 bytes) 2024-09-21T14:21:26.765 INFO:teuthology.orchestra.run.smithi073.stdout:595 write 0x9f7d582 thru 0x9f89ed3 (0xc952 bytes) 2024-09-21T14:21:26.780 INFO:teuthology.orchestra.run.smithi073.stdout:597 writesame 0xbde1e73 thru 0xbde60c1 data_size 0x23(0x424f bytes) 2024-09-21T14:21:26.780 INFO:teuthology.orchestra.run.smithi073.stdout:598 write 0x84f1ba9 thru 0x84fc21b (0xa673 bytes) 2024-09-21T14:21:26.781 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.781 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.791 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:26.794 INFO:teuthology.orchestra.run.smithi073.stdout:600 writesame 0xcbec8d7 thru 0xcbedb16 data_size 0x10(0x1240 bytes) 2024-09-21T14:21:26.801 INFO:teuthology.orchestra.run.smithi073.stdout:601 trunc from 0xcbedb17 to 0x6d7d7df 2024-09-21T14:21:26.823 INFO:teuthology.orchestra.run.smithi073.stdout:604 trunc from 0x6d7d7df to 0xba3908b 2024-09-21T14:21:26.829 INFO:teuthology.orchestra.run.smithi073.stdout:605 write 0x387613f thru 0x3878028 (0x1eea bytes) 2024-09-21T14:21:26.833 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.833 INFO:teuthology.orchestra.run.smithi073.stdout:607 read 0x99409d5 thru 0x9946dd9 (0x6405 bytes) 2024-09-21T14:21:26.833 INFO:teuthology.orchestra.run.smithi073.stdout:608 read 0x19667ac thru 0x197184a (0xb09f bytes) 2024-09-21T14:21:26.833 INFO:teuthology.orchestra.run.smithi073.stdout:609 punch from 0xb7c370d to 0xb7ca2f7, (0x6bea bytes) 2024-09-21T14:21:26.835 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:26.836+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:26.842 INFO:teuthology.orchestra.run.smithi073.stdout:610 trunc from 0xba3908b to 0x3b98f99 2024-09-21T14:21:26.854 INFO:teuthology.orchestra.run.smithi073.stdout:611 read 0x23e8055 thru 0x23ea335 (0x22e1 bytes) 2024-09-21T14:21:26.872 INFO:teuthology.orchestra.run.smithi073.stdout:612 write 0xdfe21ab thru 0xdfee643 (0xc499 bytes) 2024-09-21T14:21:26.876 INFO:teuthology.orchestra.run.smithi073.stdout:616 punch from 0x8679f37 to 0x86818ce, (0x7997 bytes) 2024-09-21T14:21:26.877 INFO:teuthology.orchestra.run.smithi073.stdout:617 trunc from 0xdfee644 to 0xcccc03b 2024-09-21T14:21:26.886 INFO:teuthology.orchestra.run.smithi073.stdout:618 writesame 0x4502b53 thru 0x4507cf2 data_size 0x8(0x51a0 bytes) 2024-09-21T14:21:26.891 INFO:teuthology.orchestra.run.smithi073.stdout:620 trunc from 0xcccc03b to 0xbb3616b 2024-09-21T14:21:26.896 INFO:teuthology.orchestra.run.smithi073.stdout:622 write 0x48dab40 thru 0x48e6069 (0xb52a bytes) 2024-09-21T14:21:26.901 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:26.901 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:26.902 INFO:teuthology.orchestra.run.smithi073.stdout:623 trunc from 0xbb3616b to 0xc944515 2024-09-21T14:21:26.907 INFO:teuthology.orchestra.run.smithi073.stdout:627 write 0x26e3329 thru 0x26ee9c8 (0xb6a0 bytes) 2024-09-21T14:21:26.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:26.911 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:26.911+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:26.914 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.914 INFO:teuthology.orchestra.run.smithi073.stdout:630 punch from 0x33e5bd9 to 0x33f327f, (0xd6a6 bytes) 2024-09-21T14:21:26.914 INFO:teuthology.orchestra.run.smithi073.stdout:631 punch from 0x92287ad to 0x9234788, (0xbfdb bytes) 2024-09-21T14:21:26.914 INFO:teuthology.orchestra.run.smithi073.stdout:632 write 0x38dbd06 thru 0x38dff93 (0x428e bytes) 2024-09-21T14:21:26.917 INFO:teuthology.orchestra.run.smithi073.stdout:633 writesame 0x9aaed7 thru 0x9b04df data_size 0x5(0x5609 bytes) 2024-09-21T14:21:26.920 INFO:teuthology.orchestra.run.smithi073.stdout:635 read 0xc90a5c0 thru 0xc90fcc9 (0x570a bytes) 2024-09-21T14:21:26.920 INFO:teuthology.orchestra.run.smithi073.stdout:636 write 0x9451e67 thru 0x9460771 (0xe90b bytes) 2024-09-21T14:21:26.929 INFO:teuthology.orchestra.run.smithi073.stdout:637 trunc from 0xc944515 to 0xe7449e4 2024-09-21T14:21:26.934 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.934 INFO:teuthology.orchestra.run.smithi073.stdout:639 read 0xf4c0b8 thru 0xf50e78 (0x4dc1 bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:640 read 0x41f4e0 thru 0x424be7 (0x5708 bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:644 read 0x5fc3ef3 thru 0x5fcf6b5 (0xb7c3 bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:645 punch from 0x9222dbd to 0x922bbec, (0x8e2f bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:647 read 0x396c2d thru 0x39f1df (0x85b3 bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:648 read 0xbe7094a thru 0xbe7d332 (0xc9e9 bytes) 2024-09-21T14:21:26.935 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.936 INFO:teuthology.orchestra.run.smithi073.stdout:651 write 0x80bf24a thru 0x80ccc49 (0xda00 bytes) 2024-09-21T14:21:26.943 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.943 INFO:teuthology.orchestra.run.smithi073.stdout:653 read 0x1bf6cfc thru 0x1bffff6 (0x92fb bytes) 2024-09-21T14:21:26.949 INFO:teuthology.orchestra.run.smithi073.stdout:654 trunc from 0xe7449e4 to 0x9103554 2024-09-21T14:21:26.968 INFO:teuthology.orchestra.run.smithi073.stdout:655 write 0xc0de7c7 thru 0xc0e2e49 (0x4683 bytes) 2024-09-21T14:21:26.979 INFO:teuthology.orchestra.run.smithi073.stdout:657 writesame 0xe3b2a98 thru 0xe3b4bb9 data_size 0x1(0x2122 bytes) 2024-09-21T14:21:26.979 INFO:teuthology.orchestra.run.smithi073.stdout:658 write 0x7145cab thru 0x7154624 (0xe97a bytes) 2024-09-21T14:21:26.983 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:26.983 INFO:teuthology.orchestra.run.smithi073.stdout:660 write 0x1d95ae4 thru 0x1da4f94 (0xf4b1 bytes) 2024-09-21T14:21:26.987 INFO:teuthology.orchestra.run.smithi073.stdout:661 write 0x1c77e95 thru 0x1c7e50e (0x667a bytes) 2024-09-21T14:21:26.998 INFO:teuthology.orchestra.run.smithi073.stdout:662 write 0x97e2378 thru 0x97e9830 (0x74b9 bytes) 2024-09-21T14:21:27.006 INFO:teuthology.orchestra.run.smithi073.stdout:663 writesame 0xd4e338c thru 0xd4f2681 data_size 0x2(0xf2f6 bytes) 2024-09-21T14:21:27.012 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:27.012+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:27.018 INFO:teuthology.orchestra.run.smithi073.stdout:664 read 0x921ec8 thru 0x92f383 (0xd4bc bytes) 2024-09-21T14:21:27.019 INFO:teuthology.orchestra.run.smithi073.stdout:665 write 0x5466472 thru 0x5470cbc (0xa84b bytes) 2024-09-21T14:21:27.026 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.033 INFO:teuthology.orchestra.run.smithi073.stdout:668 trunc from 0xe3b4bba to 0x836309d 2024-09-21T14:21:27.043 INFO:teuthology.orchestra.run.smithi073.stdout:670 writesame 0x8e0b35 thru 0x8ea158 data_size 0x6(0x9624 bytes) 2024-09-21T14:21:27.058 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.058 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.059 INFO:teuthology.orchestra.run.smithi073.stdout:672 write 0xce78b41 thru 0xce815ff (0x8abf bytes) 2024-09-21T14:21:27.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:27.068 INFO:teuthology.orchestra.run.smithi073.stdout:674 read 0x233799 thru 0x23593e (0x21a6 bytes) 2024-09-21T14:21:27.068 INFO:teuthology.orchestra.run.smithi073.stdout:676 write 0xa442ef3 thru 0xa44c3f1 (0x94ff bytes) 2024-09-21T14:21:27.077 INFO:teuthology.orchestra.run.smithi073.stdout:677 trunc from 0xce81600 to 0xd4a6647 2024-09-21T14:21:27.084 INFO:teuthology.orchestra.run.smithi073.stdout:678 trunc from 0xd4a6647 to 0xb476dd0 2024-09-21T14:21:27.092 INFO:teuthology.orchestra.run.smithi073.stdout:679 read 0x44ef91c thru 0x44f9871 (0x9f56 bytes) 2024-09-21T14:21:27.092 INFO:teuthology.orchestra.run.smithi073.stdout:680 write 0x9086a83 thru 0x909048d (0x9a0b bytes) 2024-09-21T14:21:27.097 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.097 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.103 INFO:teuthology.orchestra.run.smithi073.stdout:684 writesame 0xc603fe7 thru 0xc609dd0 data_size 0x3(0x5dea bytes) 2024-09-21T14:21:27.116 INFO:teuthology.orchestra.run.smithi073.stdout:685 trunc from 0xc609dd1 to 0x109c092 2024-09-21T14:21:27.135 INFO:teuthology.orchestra.run.smithi073.stderr:set 3.1a pg_upmap_items mapping to [4->2,7->0] 2024-09-21T14:21:27.152 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:27.153+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:27.165 INFO:teuthology.orchestra.run.smithi073.stdout:686 writesame 0xe6f3e4e thru 0xe6f8649 data_size 0x22(0x47fc bytes) 2024-09-21T14:21:27.165 INFO:teuthology.orchestra.run.smithi073.stdout:688 writesame 0xcfbd89a thru 0xcfc41af data_size 0x1(0x6916 bytes) 2024-09-21T14:21:27.177 INFO:teuthology.orchestra.run.smithi073.stdout:690 read 0x2f2763 thru 0x2fe979 (0xc217 bytes) 2024-09-21T14:21:27.178 INFO:teuthology.orchestra.run.smithi073.stdout:691 read 0xe24c150 thru 0xe24f91a (0x37cb bytes) 2024-09-21T14:21:27.178 INFO:teuthology.orchestra.run.smithi073.stdout:692 write 0x38cf122 thru 0x38d82e1 (0x91c0 bytes) 2024-09-21T14:21:27.188 INFO:teuthology.orchestra.run.smithi073.stdout:693 trunc from 0xe6f864a to 0xe75f50d 2024-09-21T14:21:27.194 INFO:teuthology.orchestra.run.smithi073.stdout:694 punch from 0xdc4fbe to 0xdc9d73, (0x4db5 bytes) 2024-09-21T14:21:27.199 INFO:teuthology.orchestra.run.smithi073.stdout:696 trunc from 0xe75f50d to 0xabb362b 2024-09-21T14:21:27.211 INFO:teuthology.orchestra.run.smithi073.stdout:697 read 0x330a0f3 thru 0x33135cf (0x94dd bytes) 2024-09-21T14:21:27.211 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.211 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.215 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:27.214+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:27.218 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.219 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.220 INFO:teuthology.orchestra.run.smithi073.stdout:700 trunc from 0xabb362b to 0x2dedc56 2024-09-21T14:21:27.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:27.234 INFO:teuthology.orchestra.run.smithi073.stdout:702 punch from 0x1a5357a to 0x1a592f4, (0x5d7a bytes) 2024-09-21T14:21:27.242 INFO:teuthology.orchestra.run.smithi073.stdout:703 writesame 0x5d5fa40 thru 0x5d65617 data_size 0x4(0x5bd8 bytes) 2024-09-21T14:21:27.243 INFO:teuthology.orchestra.run.smithi073.stdout:704 write 0x16f44bd thru 0x16fa1b3 (0x5cf7 bytes) 2024-09-21T14:21:27.251 INFO:teuthology.orchestra.run.smithi073.stdout:706 write 0x60ffc34 thru 0x6100e68 (0x1235 bytes) 2024-09-21T14:21:27.257 INFO:teuthology.orchestra.run.smithi073.stdout:707 trunc from 0x6100e69 to 0x8d12c49 2024-09-21T14:21:27.262 INFO:teuthology.orchestra.run.smithi073.stdout:708 punch from 0x6a6423 to 0x6aad29, (0x4906 bytes) 2024-09-21T14:21:27.273 INFO:teuthology.orchestra.run.smithi073.stdout:709 write 0xd1b824c thru 0xd1bfb19 (0x78ce bytes) 2024-09-21T14:21:27.289 INFO:teuthology.orchestra.run.smithi073.stdout:711 writesame 0xeaa6eea thru 0xeaa9958 data_size 0x33(0x2a6f bytes) 2024-09-21T14:21:27.289 INFO:teuthology.orchestra.run.smithi073.stdout:712 writesame 0x4a12164 thru 0x4a1eeaa data_size 0x3(0xcd47 bytes) 2024-09-21T14:21:27.299 INFO:teuthology.orchestra.run.smithi073.stdout:713 punch from 0x264ec47 to 0x265ab22, (0xbedb bytes) 2024-09-21T14:21:27.299 INFO:teuthology.orchestra.run.smithi073.stdout:714 write 0x34d4e3 thru 0x359f1f (0xca3d bytes) 2024-09-21T14:21:27.301 INFO:teuthology.orchestra.run.smithi073.stdout:715 read 0xd3476d6 thru 0xd34d0f0 (0x5a1b bytes) 2024-09-21T14:21:27.301 INFO:teuthology.orchestra.run.smithi073.stdout:716 read 0x5123bfd thru 0x512e3d8 (0xa7dc bytes) 2024-09-21T14:21:27.301 INFO:teuthology.orchestra.run.smithi073.stdout:717 punch from 0x1069176 to 0x1070cb5, (0x7b3f bytes) 2024-09-21T14:21:27.301 INFO:teuthology.orchestra.run.smithi073.stdout:718 write 0x5cd7c5d thru 0x5ce0432 (0x87d6 bytes) 2024-09-21T14:21:27.308 INFO:teuthology.orchestra.run.smithi073.stdout:720 write 0x8b4c639 thru 0x8b58bb1 (0xc579 bytes) 2024-09-21T14:21:27.312 INFO:teuthology.orchestra.run.smithi073.stdout:721 write 0x13fe6e0 thru 0x1407c92 (0x95b3 bytes) 2024-09-21T14:21:27.316 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:27.316+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:27.319 INFO:teuthology.orchestra.run.smithi073.stdout:723 trunc from 0xeaa9959 to 0xc549aa2 2024-09-21T14:21:27.340 INFO:teuthology.orchestra.run.smithi073.stdout:724 trunc from 0xc549aa2 to 0x2f00c56 2024-09-21T14:21:27.351 INFO:teuthology.orchestra.run.smithi073.stdout:725 read 0x1facf1f thru 0x1fafa2a (0x2b0c bytes) 2024-09-21T14:21:27.351 INFO:teuthology.orchestra.run.smithi073.stdout:726 punch from 0x82b4 to 0x13cf8, (0xba44 bytes) 2024-09-21T14:21:27.351 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.368 INFO:teuthology.orchestra.run.smithi073.stdout:731 writesame 0xd204e46 thru 0xd205f94 data_size 0x7(0x114f bytes) 2024-09-21T14:21:27.369 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:27.369 INFO:teuthology.orchestra.run.smithi073.stdout:733 writesame 0x871e3a1 thru 0x872293a data_size 0x3b(0x459a bytes) 2024-09-21T14:21:27.374 INFO:teuthology.orchestra.run.smithi073.stdout:734 punch from 0xa4dde11 to 0xa4e3023, (0x5212 bytes) 2024-09-21T14:21:27.374 INFO:teuthology.orchestra.run.smithi073.stdout:735 clone 4 order 19 su 4096 sc 7 2024-09-21T14:21:27.376 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.376 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:27.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:27.417+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:27.489 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.490 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:27.518 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:27.518+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:27.599 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.600 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:27.619 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:27.619+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:27.709 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.709 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:27.719 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:27.719+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:27.820 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.820 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.820 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:27.820+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:27.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:27.921 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:27.921+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:27.970 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:27.970 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:27.973 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:27.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:28.022 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:28.022+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:28.093 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:28.093 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:28.110 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:28.110 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:28.123 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:28.123+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.9820834870000006, 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029151810000000002, 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143039+0000", 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143037+0000", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143052+0000", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.449e-05 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143070+0000", 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8105e-05 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143082+0000", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2045000000000001e-05 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143128+0000", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6109999999999997e-05 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.143133+0000", 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5449999999999997e-06 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144568+0000", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001435497 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144692+0000", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012378700000000001 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.144830+0000", 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138042 2024-09-21T14:21:28.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145538+0000", 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00070826400000000001 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145872+0000", 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033409800000000001 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145918+0000", 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5388e-05 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:19.145955+0000", 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6874e-05 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.9806132520000004, 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039806640000000001, 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144510+0000", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144507+0000", 2024-09-21T14:21:28.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144519+0000", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1284000000000001e-05 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144535+0000", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144545+0000", 2024-09-21T14:21:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9110000000000005e-06 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144595+0000", 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0318999999999997e-05 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144697+0000", 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010137999999999999 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.144844+0000", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014769800000000001 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.147490+0000", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026452289999999998 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148368+0000", 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000878738 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148425+0000", 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7167999999999999e-05 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148490+0000", 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4828999999999999e-05 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.138 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.9764878760000002, 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048317120000000002, 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148635+0000", 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148634+0000", 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148643+0000", 2024-09-21T14:21:28.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8289999999999997e-06 2024-09-21T14:21:28.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148652+0000", 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.312e-06 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148659+0000", 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9800000000000001e-06 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148708+0000", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9147999999999998e-05 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.148852+0000", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014422 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.149086+0000", 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023317999999999999 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.150494+0000", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001408402 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153356+0000", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028617360000000001 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153421+0000", 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5625999999999998e-05 2024-09-21T14:21:28.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:20.153467+0000", 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5402999999999999e-05 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1411 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.541438061, 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026751169999999999, 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1411 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583685+0000", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583684+0000", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583688+0000", 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7239999999999998e-06 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583696+0000", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1440000000000003e-06 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583701+0000", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.527e-06 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583733+0000", 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2721000000000001e-05 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583786+0000", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2355000000000003e-05 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.583940+0000", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015385 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.584953+0000", 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010138580000000001 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586305+0000", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013520089999999999 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586336+0000", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0846999999999998e-05 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.586360+0000", 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3689999999999998e-05 2024-09-21T14:21:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1741 3.1a 3:5846146e:::rbd_object_map.11212f271376:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.4715465820000002, 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030142189999999998, 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1741 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653576+0000", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653575+0000", 2024-09-21T14:21:28.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653581+0000", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3399999999999997e-06 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653591+0000", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0144e-05 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653596+0000", 2024-09-21T14:21:28.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9470000000000002e-06 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653627+0000", 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1130999999999997e-05 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653678+0000", 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1227000000000003e-05 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.653887+0000", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020935800000000001 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.654611+0000", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072371200000000005 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656495+0000", 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018841280000000001 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656550+0000", 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4264000000000001e-05 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656591+0000", 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0929000000000001e-05 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:1742 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2469136~21232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T14:21:28.156 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.4684719040000003, 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041457020000000002, 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1742 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656651+0000", 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656650+0000", 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656674+0000", 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4219999999999999e-05 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656683+0000", 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7479999999999999e-06 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656688+0000", 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7119999999999999e-06 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656733+0000", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4846999999999999e-05 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.656871+0000", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137933 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.657079+0000", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020842699999999999 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.657873+0000", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079421600000000004 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660706+0000", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028326319999999999 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660759+0000", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3529999999999997e-05 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:23.660797+0000", 2024-09-21T14:21:28.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7178000000000002e-05 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2030 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.8969983699999999, 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052215689999999997, 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2030 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228124+0000", 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228123+0000", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228129+0000", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1190000000000002e-06 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228144+0000", 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4198e-05 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228151+0000", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4789999999999999e-06 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228209+0000", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8032999999999998e-05 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228300+0000", 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0861999999999998e-05 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.228494+0000", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019448999999999999 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.231566+0000", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003072052 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233247+0000", 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016804929999999999 2024-09-21T14:21:28.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233303+0000", 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6005999999999999e-05 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.233346+0000", 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3072999999999999e-05 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3388 3.a e26/19 3:5752cc72:::rbd_data.1121cf46bdbd.00000000000000e8:head v 26'52, mlcod=26'50)", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.063691+0000", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.061431389, 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032594400000000002, 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063691+0000", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063691+0000", 2024-09-21T14:21:28.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063690+0000", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063700+0000", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7219999999999994e-06 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063719+0000", 2024-09-21T14:21:28.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8771999999999999e-05 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063728+0000", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2750000000000001e-06 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063775+0000", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7429000000000003e-05 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063791+0000", 2024-09-21T14:21:28.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.562e-05 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.066878+0000", 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030864849999999999 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.066951+0000", 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3349000000000005e-05 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3392 3.a e26/19 3:537e4482:::rbd_data.1121cf46bdbd.0000000000000182:head v 26'53, mlcod=26'50)", 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.063843+0000", 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.0612797839999999, 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0058709210000000003, 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063843+0000", 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063843+0000", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063842+0000", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063848+0000", 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9270000000000002e-06 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063858+0000", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063864+0000", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7899999999999996e-06 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.063993+0000", 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012907399999999999 2024-09-21T14:21:28.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.064012+0000", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9176000000000001e-05 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.069689+0000", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0056768640000000002 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.069714+0000", 2024-09-21T14:21:28.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899e-05 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3577 3.1f e26/19 3:fafb0fb5:::rbd_data.1121cf46bdbd.000000000000005f:head v 26'3, mlcod=22'1)", 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.128272+0000", 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9968512990000002, 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045603099999999997, 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128272+0000", 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128272+0000", 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128271+0000", 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128278+0000", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3209999999999999e-06 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128288+0000", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.013e-05 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128294+0000", 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0079999999999999e-06 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128337+0000", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2629999999999997e-05 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.128350+0000", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3363000000000001e-05 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.132796+0000", 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044456130000000002 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.132832+0000", 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6186000000000002e-05 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3580 3.d e26/19 3:b7099a91:::rbd_data.1121cf46bdbd.000000000000008f:head v 26'7, mlcod=26'5)", 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.129376+0000", 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9957465559999998, 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0034784669999999998, 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129376+0000", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129376+0000", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129376+0000", 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129381+0000", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4049999999999999e-06 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129391+0000", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7720000000000006e-06 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129397+0000", 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:21:28.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129440+0000", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3127000000000002e-05 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.129462+0000", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2002e-05 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.132811+0000", 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033490809999999998 2024-09-21T14:21:28.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.132855+0000", 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3856999999999997e-05 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3582 3.19 e26/19 3:9f1fadb7:::rbd_data.1121cf46bdbd.00000000000000d9:head v 26'32, mlcod=26'30)", 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.130610+0000", 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9945124019999998, 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004826192, 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.184 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130610+0000", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130610+0000", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130610+0000", 2024-09-21T14:21:28.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130616+0000", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1659999999999999e-06 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130626+0000", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7880000000000001e-06 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130632+0000", 2024-09-21T14:21:28.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1279999999999996e-06 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130673+0000", 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.155e-05 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130688+0000", 2024-09-21T14:21:28.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4528e-05 2024-09-21T14:21:28.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.135398+0000", 2024-09-21T14:21:28.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0047098499999999998 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.135437+0000", 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8863999999999999e-05 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3645 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:6 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.141884+0000", 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9832391949999999, 2024-09-21T14:21:28.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002864222, 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3645 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141884+0000", 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141884+0000", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141883+0000", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141890+0000", 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141896+0000", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4779999999999996e-06 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141901+0000", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3000000000000003e-06 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144562+0000", 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026613940000000001 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144642+0000", 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0240000000000004e-05 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144748+0000", 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105586 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3646 3.10 3:0a4db0b1:::rbd_data.11212f271376.0000000000000004:6 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.141905+0000", 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9832177280000001, 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003025083, 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3646 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141905+0000", 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141905+0000", 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141905+0000", 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141914+0000", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141921+0000", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7800000000000003e-06 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.141926+0000", 2024-09-21T14:21:28.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7439999999999998e-06 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144767+0000", 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028416560000000001 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144846+0000", 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8818000000000003e-05 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.144930+0000", 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3948000000000003e-05 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4134 3.1a 3:59cd9250:::rbd_data.1121cf46bdbd.0000000000000066:head [set-alloc-hint object_size 524288 write_size 524288,write 468921~6215] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.659580+0000", 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.4655432959999999, 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002745197, 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4134 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659580+0000", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659580+0000", 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659578+0000", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659594+0000", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6217000000000001e-05 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659608+0000", 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4099e-05 2024-09-21T14:21:28.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659615+0000", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7440000000000001e-06 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659659+0000", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4367999999999999e-05 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659759+0000", 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9968000000000002e-05 2024-09-21T14:21:28.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659846+0000", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7099000000000001e-05 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.660684+0000", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083749400000000004 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662237+0000", 2024-09-21T14:21:28.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015534629999999999 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662284+0000", 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6937999999999997e-05 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662325+0000", 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0815000000000001e-05 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4144 3.1a 3:59cd9250:::rbd_data.1121cf46bdbd.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.685160+0000", 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.439962892, 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028819179999999998, 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4144 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685160+0000", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685160+0000", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685158+0000", 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685168+0000", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685186+0000", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8505e-05 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685196+0000", 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8030000000000008e-06 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685252+0000", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5930999999999997e-05 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685332+0000", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0187999999999999e-05 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.685441+0000", 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010917999999999999 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.686534+0000", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010927790000000001 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.687954+0000", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014201940000000001 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.688006+0000", 2024-09-21T14:21:28.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1842000000000003e-05 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.688042+0000", 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5836e-05 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:4234 3.1f e26/19 3:f8614e23:::rbd_data.1121cf46bdbd.000000000000016f:head v 26'5, mlcod=26'3)", 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.774056+0000", 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.351066613, 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004697634, 2024-09-21T14:21:28.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774056+0000", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774056+0000", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774055+0000", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774078+0000", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2344000000000002e-05 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774088+0000", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0671e-05 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774093+0000", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9139999999999999e-06 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774130+0000", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6831e-05 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.774152+0000", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1627000000000001e-05 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778716+0000", 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045646560000000003 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778754+0000", 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:4254 3.19 e26/19 3:98646315:::rbd_data.1121cf46bdbd.000000000000016e:head v 26'37, mlcod=26'35)", 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.807122+0000", 2024-09-21T14:21:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.3180005239999999, 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003167043, 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807122+0000", 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807122+0000", 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807122+0000", 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807130+0000", 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1200000000000002e-06 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807141+0000", 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1562999999999999e-05 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807148+0000", 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.426e-06 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807255+0000", 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010699799999999999 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807276+0000", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1104e-05 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810237+0000", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029612760000000001 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810289+0000", 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2321000000000003e-05 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:4255 3.1f e26/19 3:f8614e23:::rbd_data.1121cf46bdbd.000000000000016f:head v 26'6, mlcod=26'4)", 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.807278+0000", 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.3178445729999999, 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030438679999999999, 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807278+0000", 2024-09-21T14:21:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807278+0000", 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807277+0000", 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807283+0000", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6269999999999997e-06 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807292+0000", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2080000000000006e-06 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807298+0000", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9050000000000002e-06 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807363+0000", 2024-09-21T14:21:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4467000000000005e-05 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807383+0000", 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9839000000000001e-05 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810293+0000", 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002910667 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810322+0000", 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8965000000000001e-05 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:4253 3.f e26/19 3:f369e7fb:::rbd_data.1121cf46bdbd.000000000000011e:head v 26'33, mlcod=26'30)", 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.807373+0000", 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.3177502209999998, 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029746550000000001, 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807373+0000", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807373+0000", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807372+0000", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807377+0000", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7639999999999996e-06 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807385+0000", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6319999999999997e-06 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807391+0000", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807542+0000", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015068300000000001 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.807552+0000", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0491e-05 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810324+0000", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027722620000000002 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.810347+0000", 2024-09-21T14:21:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2994e-05 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:28.227 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:28.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:28.229 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:28.229 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:28.229 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:28.229 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:28.230 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:28.230 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:28.230 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:28.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:21:28.231 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:28.231 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:28.232 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:28.233 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:28.232+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:28.254 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone3 from 0xd205f95 (overlap 0x4f8eeb) to 0x0 2024-09-21T14:21:28.329 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T14:21:28.332 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:28.332+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:28.334 INFO:teuthology.orchestra.run.smithi073.stdout:736 trunc from 0xd205f95 to 0x99b5c89 2024-09-21T14:21:28.354 INFO:teuthology.orchestra.run.smithi073.stdout:738 punch from 0x3d48d28 to 0x3d49c3f, (0xf17 bytes) 2024-09-21T14:21:28.363 INFO:teuthology.orchestra.run.smithi073.stdout:740 write 0xacc379f thru 0xacc63df (0x2c41 bytes) 2024-09-21T14:21:28.377 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:28.377 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:28.377 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:28.377 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:28.380 INFO:teuthology.orchestra.run.smithi073.stdout:741 write 0xcab8c58 thru 0xcabc4fa (0x38a3 bytes) 2024-09-21T14:21:28.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:28.392 INFO:teuthology.orchestra.run.smithi073.stdout:742 write 0x971fdb9 thru 0x972fd30 (0xff78 bytes) 2024-09-21T14:21:28.395 INFO:teuthology.orchestra.run.smithi073.stdout:743 trunc from 0xcabc4fb to 0xc5a03c8 2024-09-21T14:21:28.408 INFO:teuthology.orchestra.run.smithi073.stdout:744 punch from 0x239dac0 to 0x23a2ea2, (0x53e2 bytes) 2024-09-21T14:21:28.409 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.409 INFO:teuthology.orchestra.run.smithi073.stdout:746 punch from 0xb2826cf to 0xb28b964, (0x9295 bytes) 2024-09-21T14:21:28.417 INFO:teuthology.orchestra.run.smithi073.stdout:747 write 0xd510e46 thru 0xd51f917 (0xead2 bytes) 2024-09-21T14:21:28.429 INFO:teuthology.orchestra.run.smithi073.stdout:749 write 0x28370fc thru 0x2843d88 (0xcc8d bytes) 2024-09-21T14:21:28.434 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:28.434+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:28.442 INFO:teuthology.orchestra.run.smithi073.stdout:751 punch from 0x87c9e71 to 0x87ccceb, (0x2e7a bytes) 2024-09-21T14:21:28.448 INFO:teuthology.orchestra.run.smithi073.stdout:752 trunc from 0xd51f918 to 0x8027362 2024-09-21T14:21:28.475 INFO:teuthology.orchestra.run.smithi073.stdout:753 read 0x2fa25b6 thru 0x2fabb37 (0x9582 bytes) 2024-09-21T14:21:28.475 INFO:teuthology.orchestra.run.smithi073.stdout:754 read 0x5136918 thru 0x513e3e8 (0x7ad1 bytes) 2024-09-21T14:21:28.475 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.480 INFO:teuthology.orchestra.run.smithi073.stdout:756 trunc from 0x8027362 to 0x483658e 2024-09-21T14:21:28.501 INFO:teuthology.orchestra.run.smithi073.stdout:758 read 0x997eb8 thru 0x9a7219 (0xf362 bytes) 2024-09-21T14:21:28.503 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.503 INFO:teuthology.orchestra.run.smithi073.stdout:760 read 0x1671291 thru 0x1671810 (0x580 bytes) 2024-09-21T14:21:28.503 INFO:teuthology.orchestra.run.smithi073.stdout:761 read 0x17cc78e thru 0x17d76f5 (0xaf68 bytes) 2024-09-21T14:21:28.514 INFO:teuthology.orchestra.run.smithi073.stdout:762 trunc from 0x483658e to 0xd14b863 2024-09-21T14:21:28.522 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:28.522 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:28.522 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:28.522 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:28.522 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:28.526 INFO:teuthology.orchestra.run.smithi073.stdout:763 write 0xde2fcd8 thru 0xde3048b (0x7b4 bytes) 2024-09-21T14:21:28.530 INFO:teuthology.orchestra.run.smithi073.stdout:764 read 0x4ef7602 thru 0x4ef9fbe (0x29bd bytes) 2024-09-21T14:21:28.530 INFO:teuthology.orchestra.run.smithi073.stdout:765 read 0x5825429 thru 0x582687f (0x1457 bytes) 2024-09-21T14:21:28.530 INFO:teuthology.orchestra.run.smithi073.stdout:766 read 0x3ad200f thru 0x3ae0b4f (0xeb41 bytes) 2024-09-21T14:21:28.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:28.534 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:28.535+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:28.544 INFO:teuthology.orchestra.run.smithi073.stdout:768 trunc from 0xde3048c to 0x26071e7 2024-09-21T14:21:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:769 write 0x4839226 thru 0x483a889 (0x1664 bytes) 2024-09-21T14:21:28.597 INFO:teuthology.orchestra.run.smithi073.stdout:770 write 0x64a601c thru 0x64aa9df (0x49c4 bytes) 2024-09-21T14:21:28.608 INFO:teuthology.orchestra.run.smithi073.stdout:774 punch from 0x39cac40 to 0x39cc771, (0x1b31 bytes) 2024-09-21T14:21:28.608 INFO:teuthology.orchestra.run.smithi073.stdout:775 read 0x613e5bd thru 0x613f3bc (0xe00 bytes) 2024-09-21T14:21:28.608 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.619 INFO:teuthology.orchestra.run.smithi073.stdout:777 writesame 0xac4df8a thru 0xac4e963 data_size 0xd(0x9da bytes) 2024-09-21T14:21:28.619 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.620 INFO:teuthology.orchestra.run.smithi073.stdout:779 read 0x5d41b0d thru 0x5d43af7 (0x1feb bytes) 2024-09-21T14:21:28.620 INFO:teuthology.orchestra.run.smithi073.stdout:780 writesame 0xa83ed2 thru 0xa903a6 data_size 0x1(0xc4d5 bytes) 2024-09-21T14:21:28.636 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:28.636+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:28.640 INFO:teuthology.orchestra.run.smithi073.stdout:781 write 0xbec743e thru 0xbecb009 (0x3bcc bytes) 2024-09-21T14:21:28.650 INFO:teuthology.orchestra.run.smithi073.stdout:782 read 0x735b32e thru 0x7361675 (0x6348 bytes) 2024-09-21T14:21:28.655 INFO:teuthology.orchestra.run.smithi073.stdout:783 write 0xcc63cf5 thru 0xcc64097 (0x3a3 bytes) 2024-09-21T14:21:28.660 INFO:teuthology.orchestra.run.smithi073.stdout:784 write 0x8fd0322 thru 0x8fdfdfe (0xfadd bytes) 2024-09-21T14:21:28.672 INFO:teuthology.orchestra.run.smithi073.stdout:786 write 0x12480ee thru 0x1251463 (0x9376 bytes) 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3167 3.5 3:a0fa18b1:::rbd_data.1121cf46bdbd.00000000000000e7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.88793057, 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053239749999999999, 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3167 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784865+0000", 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784864+0000", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784870+0000", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3620000000000002e-06 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784883+0000", 2024-09-21T14:21:28.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2839e-05 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784890+0000", 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5070000000000003e-06 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.784942+0000", 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1490000000000003e-05 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785057+0000", 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115359 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785161+0000", 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104171 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.785979+0000", 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081793100000000004 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790095+0000", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041157499999999996 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790147+0000", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.223e-05 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.790189+0000", 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1365999999999997e-05 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3566 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.101735+0000", 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.571059838, 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006701261, 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3566 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101735+0000", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101735+0000", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101734+0000", 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101741+0000", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.725e-06 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101756+0000", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4447000000000001e-05 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101763+0000", 2024-09-21T14:21:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4359999999999996e-06 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101863+0000", 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9759999999999994e-05 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.101953+0000", 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0030000000000004e-05 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.102373+0000", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000420209 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.105101+0000", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027279190000000001 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.108331+0000", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032301019999999999 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.108395+0000", 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4109999999999994e-05 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.108437+0000", 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1616000000000003e-05 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3581 3.2 e26/19 3:432a3115:::rbd_data.1121cf46bdbd.00000000000000d8:head v 26'21, mlcod=26'19)", 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.130275+0000", 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.54252075, 2024-09-21T14:21:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053820559999999996, 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130275+0000", 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130275+0000", 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130274+0000", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130279+0000", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6110000000000001e-06 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130289+0000", 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5000000000000005e-06 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130295+0000", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9290000000000003e-06 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130335+0000", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9638999999999998e-05 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.130348+0000", 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3755e-05 2024-09-21T14:21:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.135618+0000", 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052702189999999996 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.135657+0000", 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8127000000000003e-05 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3585 3.16 3:685e4703:::rbd_data.1121cf46bdbd.00000000000000e2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.133241+0000", 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.539554775, 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005287974, 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3585 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133241+0000", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133241+0000", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133239+0000", 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133248+0000", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0180000000000007e-06 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133261+0000", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.305e-05 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133266+0000", 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:21:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133309+0000", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3133999999999997e-05 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133366+0000", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6944999999999998e-05 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133434+0000", 2024-09-21T14:21:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7744999999999996e-05 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.135857+0000", 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024238300000000001 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.138438+0000", 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025804510000000001 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.138489+0000", 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0856999999999998e-05 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.138529+0000", 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9786999999999997e-05 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:3586 3.2 e26/19 3:4375256b:::rbd_data.1121cf46bdbd.00000000000000e3:head v 26'22, mlcod=26'19)", 2024-09-21T14:21:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.133522+0000", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.5392730540000001, 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049488589999999999, 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133522+0000", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133522+0000", 2024-09-21T14:21:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133521+0000", 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133529+0000", 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2470000000000002e-06 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133550+0000", 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1098999999999999e-05 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133559+0000", 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5200000000000003e-06 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133597+0000", 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8012999999999997e-05 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.133615+0000", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7476e-05 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.138447+0000", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048320969999999996 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.138471+0000", 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4210999999999999e-05 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3714 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.167287+0000", 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.5055079419999999, 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005362276, 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3714 2024-09-21T14:21:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167287+0000", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167287+0000", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167287+0000", 2024-09-21T14:21:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167290+0000", 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4910000000000001e-06 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167297+0000", 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1430000000000003e-06 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.167301+0000", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.168898+0000", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015968429999999999 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.168974+0000", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6019000000000005e-05 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.169337+0000", 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000363252 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.170510+0000", 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001173077 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.172556+0000", 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020452909999999999 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.172606+0000", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0210000000000002e-05 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.172650+0000", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3702999999999998e-05 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3833 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.326226+0000", 2024-09-21T14:21:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.3465688359999999, 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049928680000000001, 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3833 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326226+0000", 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326226+0000", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326226+0000", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326230+0000", 2024-09-21T14:21:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0840000000000002e-06 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326238+0000", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650000000000003e-06 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.326243+0000", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3909999999999997e-06 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.327050+0000", 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00080725600000000001 2024-09-21T14:21:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.327132+0000", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1392999999999998e-05 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.327481+0000", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034904400000000001 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.330806+0000", 2024-09-21T14:21:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033255720000000002 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.331136+0000", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032971699999999999 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.331184+0000", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7626000000000001e-05 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.331219+0000", 2024-09-21T14:21:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5627999999999998e-05 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3838 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.332823+0000", 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.3399719220000001, 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051385199999999997, 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3838 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332823+0000", 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332823+0000", 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332823+0000", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332826+0000", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1990000000000002e-06 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332832+0000", 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2249999999999997e-06 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.332837+0000", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.334775+0000", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019380039999999999 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.334861+0000", 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5736999999999996e-05 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.335278+0000", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041755399999999999 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.337488+0000", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022101619999999999 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.337909+0000", 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00042114000000000002 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.337942+0000", 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2085000000000002e-05 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.337962+0000", 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0395999999999999e-05 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3846 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.343775+0000", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.3290202020000002, 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092752749999999995, 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3846 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343775+0000", 2024-09-21T14:21:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343775+0000", 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343775+0000", 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343778+0000", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8310000000000001e-06 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343783+0000", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5289999999999999e-06 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.343787+0000", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2169999999999996e-06 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.345026+0000", 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012385740000000001 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.345116+0000", 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9718999999999999e-05 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.345644+0000", 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00052843400000000004 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.347276+0000", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016320639999999999 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.352911+0000", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0056344979999999999 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.352962+0000", 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1761999999999997e-05 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.353050+0000", 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8026000000000001e-05 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:3848 3.16 3:6c0065ea:::rbd_data.1121cf46bdbd.000000000000012a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.346526+0000", 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.3262695739999999, 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065881969999999996, 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 3848 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346526+0000", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346526+0000", 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346524+0000", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346532+0000", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0999999999999998e-06 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346544+0000", 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2551000000000001e-05 2024-09-21T14:21:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346552+0000", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7009999999999996e-06 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346590+0000", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858000000000002e-05 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.346669+0000", 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9629000000000006e-05 2024-09-21T14:21:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3", 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.347677+0000", 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001007422 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.352940+0000", 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052631309999999999 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.353056+0000", 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116539 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.353080+0000", 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3870999999999999e-05 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.353114+0000", 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3667000000000001e-05 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4014 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.528132+0000", 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1446637649999998, 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052973109999999999, 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4014 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528132+0000", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528132+0000", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528131+0000", 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528135+0000", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.631e-06 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528141+0000", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6200000000000001e-06 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528147+0000", 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2429999999999996e-06 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528809+0000", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00066258099999999998 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528902+0000", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3234999999999998e-05 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.529318+0000", 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041514200000000002 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533269+0000", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039517019999999996 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533362+0000", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3105999999999999e-05 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533391+0000", 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9079999999999999e-05 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533429+0000", 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7441999999999998e-05 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4015 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.528154+0000", 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1446414690000002, 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0056106489999999997, 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4015 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528154+0000", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528154+0000", 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528153+0000", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528156+0000", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6599999999999999e-06 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528162+0000", 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.528166+0000", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.493e-06 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.529721+0000", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015548929999999999 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.529814+0000", 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.530267+0000", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00045364500000000002 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533549+0000", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032815629999999999 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533713+0000", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016404700000000001 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533743+0000", 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0450000000000002e-05 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.533764+0000", 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1027999999999999e-05 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4094 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.614182+0000", 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.0586136399999999, 2024-09-21T14:21:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068114250000000003, 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4094 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614182+0000", 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614182+0000", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614181+0000", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614185+0000", 2024-09-21T14:21:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6710000000000001e-06 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614191+0000", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5710000000000003e-06 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.614196+0000", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.615736+0000", 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015396839999999999 2024-09-21T14:21:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.615800+0000", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3502000000000003e-05 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.616022+0000", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222492 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.620708+0000", 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0046861180000000004 2024-09-21T14:21:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.620911+0000", 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202435 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.620953+0000", 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2734000000000001e-05 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.620993+0000", 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9672000000000002e-05 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4132 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.656747+0000", 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.0160482000000002, 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0055380109999999998, 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4132 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656747+0000", 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656747+0000", 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656747+0000", 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656750+0000", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6480000000000001e-06 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656758+0000", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4420000000000001e-06 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656764+0000", 2024-09-21T14:21:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1770000000000003e-06 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.657369+0000", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00060532700000000001 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.657457+0000", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7738999999999995e-05 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.657744+0000", 2024-09-21T14:21:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028666199999999997 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.659234+0000", 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001490601 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662201+0000", 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029663850000000002 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662248+0000", 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6998e-05 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662285+0000", 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7530000000000002e-05 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4133 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.656773+0000", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.0160228060000001, 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0056536729999999997, 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4133 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656773+0000", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656773+0000", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656772+0000", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656776+0000", 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.597e-06 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656782+0000", 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1380000000000004e-06 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.656786+0000", 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5739999999999997e-06 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.658077+0000", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012906969999999999 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.658163+0000", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6080000000000003e-05 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.658377+0000", 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214277 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662074+0000", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036964469999999998 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662335+0000", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026096700000000002 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662387+0000", 2024-09-21T14:21:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2386000000000003e-05 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.662426+0000", 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8998999999999997e-05 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4213 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.751748+0000", 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.921047261, 2024-09-21T14:21:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0050575840000000004, 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4213 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751748+0000", 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751748+0000", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751747+0000", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751751+0000", 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:21:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751758+0000", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7009999999999999e-06 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.751763+0000", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3369999999999999e-06 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.753396+0000", 2024-09-21T14:21:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001632203 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.753481+0000", 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5439999999999995e-05 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.753740+0000", 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025869599999999999 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.756394+0000", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026538870000000002 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.756718+0000", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032446999999999999 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.756769+0000", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0955999999999998e-05 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.756806+0000", 2024-09-21T14:21:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6529999999999998e-05 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4232 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.769112+0000", 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9036834300000001, 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0097695230000000004, 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4232 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769112+0000", 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769112+0000", 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769112+0000", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769114+0000", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.819e-06 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769122+0000", 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4229999999999999e-06 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.769130+0000", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4109999999999996e-06 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.771001+0000", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018702390000000001 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.771093+0000", 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2e-05 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.771503+0000", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041018499999999998 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778677+0000", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0071741030000000003 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778819+0000", 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014216200000000001 2024-09-21T14:21:28.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778853+0000", 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4474000000000002e-05 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.778881+0000", 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7968999999999999e-05 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.767 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:4282 3.e 3:70f2e685:::rbd_object_map.1121cf46bdbd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:25.843085+0000", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.8297107600000002, 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068141130000000001, 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4282 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843085+0000", 2024-09-21T14:21:28.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843085+0000", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843084+0000", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843092+0000", 2024-09-21T14:21:28.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4590000000000001e-06 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843099+0000", 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3180000000000001e-06 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.843103+0000", 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3830000000000002e-06 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.844698+0000", 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001594921 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.844786+0000", 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7284e-05 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.845063+0000", 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00027744 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.849778+0000", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0047152399999999999 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.849816+0000", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7706000000000001e-05 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.849862+0000", 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6329999999999999e-05 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:25.849899+0000", 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6409000000000003e-05 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6310 3.5 3:a7bfb709:::rbd_data.11214feba615.0000000000000132:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.451949+0000", 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.220846563, 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0066502549999999999, 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:28.773 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6310 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451949+0000", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451949+0000", 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451948+0000", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451952+0000", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8929999999999998e-06 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451961+0000", 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.363e-06 2024-09-21T14:21:28.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.451967+0000", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4530000000000001e-06 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.452024+0000", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7161999999999999e-05 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.452106+0000", 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2436999999999997e-05 2024-09-21T14:21:28.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.452210+0000", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000103729 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.454122+0000", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001911315 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.458504+0000", 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043829619999999998 2024-09-21T14:21:28.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.458557+0000", 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2839e-05 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.458599+0000", 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1669999999999999e-05 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:6516 3.1 e29/19 3:85470541:::rbd_data.11214feba615.0000000000000015:head v 29'23, mlcod=29'21)", 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.625440+0000", 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.047355500000000002, 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0099036590000000004, 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625440+0000", 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625440+0000", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625438+0000", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625624+0000", 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018560899999999999 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625643+0000", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8977000000000001e-05 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625652+0000", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7660000000000004e-06 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625700+0000", 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8313000000000003e-05 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.625722+0000", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1610000000000001e-05 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.635309+0000", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009586809 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.635343+0000", 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4897999999999997e-05 2024-09-21T14:21:28.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.783 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:28.783 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:28.783 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:28.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:28.786 INFO:teuthology.orchestra.run.smithi073.stdout:787 writesame 0x7335337 thru 0x733c3db data_size 0x1(0x70a5 bytes) 2024-09-21T14:21:28.786 INFO:teuthology.orchestra.run.smithi073.stdout:788 trunc from 0xcc64098 to 0x980b855 2024-09-21T14:21:28.786 INFO:teuthology.orchestra.run.smithi073.stdout:789 writesame 0x1b94185 thru 0x1b945c7 data_size 0x1(0x443 bytes) 2024-09-21T14:21:28.786 INFO:teuthology.orchestra.run.smithi073.stdout:795 write 0x340ecbd thru 0x3410aa2 (0x1de6 bytes) 2024-09-21T14:21:28.786 INFO:teuthology.orchestra.run.smithi073.stdout:796 read 0x3bc0b00 thru 0x3bcfdcd (0xf2ce bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:797 write 0x9f061ab thru 0x9f124b7 (0xc30d bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:798 write 0x86c087b thru 0x86c3ee1 (0x3667 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:799 punch from 0x1239290 to 0x12416f2, (0x8462 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:802 write 0xde35f2e thru 0xde3738f (0x1462 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:805 read 0x927be6e thru 0x9285995 (0x9b28 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:806 read 0x447ca74 thru 0x44824e2 (0x5a6f bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:807 read 0x6803861 thru 0x68111c4 (0xd964 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:808 punch from 0x2dc9366 to 0x2dcc4a5, (0x313f bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:809 read 0xda8734a thru 0xda89d35 (0x29ec bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:810 read 0x564cefa thru 0x565b726 (0xe82d bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:811 write 0x9e0a0a4 thru 0x9e1281f (0x877c bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:812 punch from 0x1cb835c to 0x1cbfc54, (0x78f8 bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:813 read 0x9158cc2 thru 0x916867b (0xf9ba bytes) 2024-09-21T14:21:28.787 INFO:teuthology.orchestra.run.smithi073.stdout:814 trunc from 0xde37390 to 0x78e129f 2024-09-21T14:21:28.788 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:28.784+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:28.798 INFO:teuthology.orchestra.run.smithi073.stdout:816 writesame 0x2ecce27 thru 0x2ece007 data_size 0x1(0x11e1 bytes) 2024-09-21T14:21:28.806 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:28.806 INFO:teuthology.orchestra.run.smithi073.stdout:818 clone 5 order 18 su 2048 sc 7 2024-09-21T14:21:28.885 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:28.885+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:28.900 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:28.900 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:28.900 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:28.900 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:28.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:28.985 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:28.986+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:29.051 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:29.051 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:29.051 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:29.051 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:29.051 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:29.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:29.103 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:29.103+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:29.162 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:29.162 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:29.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:29.162 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.619718351, 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.163 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:29.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:29.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:29.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2556 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9233643069999999, 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014254843, 2024-09-21T14:21:29.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2556 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238293+0000", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238295+0000", 2024-09-21T14:21:29.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238301+0000", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7209999999999999e-06 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238303+0000", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3999999999999999e-06 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252353+0000", 2024-09-21T14:21:29.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014049918999999999 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252438+0000", 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5140000000000001e-05 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252548+0000", 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110009 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2557 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18153472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9233513440000003, 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014393985, 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2557 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238306+0000", 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238308+0000", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8339999999999999e-06 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238313+0000", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1880000000000001e-06 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238317+0000", 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0910000000000003e-06 2024-09-21T14:21:29.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252584+0000", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014267334 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252642+0000", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8162000000000003e-05 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252700+0000", 2024-09-21T14:21:29.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7827e-05 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2558 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18481152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9233371259999998, 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014618236999999999, 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.178 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2558 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238321+0000", 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238320+0000", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238322+0000", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8700000000000001e-06 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238327+0000", 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238331+0000", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8210000000000003e-06 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252746+0000", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014415156 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252830+0000", 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3974000000000006e-05 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252939+0000", 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108448 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2559 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18808832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9233225569999997, 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014758411000000001, 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.182 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2559 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238335+0000", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238337+0000", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7290000000000001e-06 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238342+0000", 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4709999999999998e-06 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.238346+0000", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9310000000000001e-06 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.252963+0000", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014616878999999999 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.253026+0000", 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3516000000000006e-05 2024-09-21T14:21:29.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.253094+0000", 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7224000000000006e-05 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2742 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9183553409999998, 2024-09-21T14:21:29.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014172158000000001, 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2742 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243302+0000", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243305+0000", 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.779e-06 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243311+0000", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.19e-06 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243316+0000", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4890000000000003e-06 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257345+0000", 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014028491000000001 2024-09-21T14:21:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257410+0000", 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5575999999999997e-05 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257475+0000", 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4015000000000003e-05 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2743 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9183368959999996, 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014360362999999999, 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2743 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243321+0000", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243320+0000", 2024-09-21T14:21:29.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243323+0000", 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.492e-06 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243333+0000", 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5990000000000008e-06 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243337+0000", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.42e-06 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257514+0000", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014177286000000001 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257583+0000", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9256000000000002e-05 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257681+0000", 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7678999999999998e-05 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2744 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9183164850000001, 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014566601, 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2744 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243341+0000", 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243343+0000", 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4449999999999999e-06 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243350+0000", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8929999999999999e-06 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243355+0000", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4789999999999998e-06 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257726+0000", 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014371754 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257815+0000", 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9054000000000005e-05 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257908+0000", 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2342999999999993e-05 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.199 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2745 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9182982939999995, 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014728415, 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2745 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243359+0000", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243361+0000", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3379999999999999e-06 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243368+0000", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9670000000000004e-06 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243373+0000", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0150000000000003e-06 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.257934+0000", 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014560518 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258022+0000", 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8175000000000002e-05 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258088+0000", 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5741999999999995e-05 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2746 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.918279944, 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014896428, 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2746 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243378+0000", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243377+0000", 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243399+0000", 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1429000000000001e-05 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243406+0000", 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8510000000000001e-06 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243411+0000", 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1669999999999998e-06 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258108+0000", 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014697306 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258185+0000", 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6841000000000005e-05 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258274+0000", 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9184999999999994e-05 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2747 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9182419660000001, 2024-09-21T14:21:29.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015034008999999999, 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2747 2024-09-21T14:21:29.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243416+0000", 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243415+0000", 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243418+0000", 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243424+0000", 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7970000000000003e-06 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243446+0000", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2084000000000001e-05 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258310+0000", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014864007 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258389+0000", 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9016000000000004e-05 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258450+0000", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0309000000000001e-05 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2748 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9182063869999997, 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015218272, 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2748 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243451+0000", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243454+0000", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7310000000000002e-06 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243460+0000", 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243464+0000", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4039999999999997e-06 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258488+0000", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015024084 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258569+0000", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1258e-05 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258670+0000", 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100163 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2749 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9181894000000002, 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015336407, 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2749 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243468+0000", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243474+0000", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9580000000000004e-06 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243481+0000", 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6719999999999998e-06 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243494+0000", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2979999999999999e-05 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258712+0000", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015217965 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258754+0000", 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2005999999999997e-05 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258805+0000", 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1184000000000003e-05 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2750 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9181266709999996, 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015489749000000001, 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2750 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243531+0000", 2024-09-21T14:21:29.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243530+0000", 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243535+0000", 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.42e-06 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243543+0000", 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1810000000000001e-06 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243549+0000", 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2090000000000002e-06 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258838+0000", 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015288652999999999 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.258914+0000", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5971999999999997e-05 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259021+0000", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010700399999999999 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2751 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 16908288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:29.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9181036210000002, 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015592231, 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2751 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243554+0000", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243556+0000", 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4870000000000001e-06 2024-09-21T14:21:29.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243562+0000", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7019999999999998e-06 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243569+0000", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5900000000000002e-06 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259045+0000", 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015475668 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259095+0000", 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9656e-05 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259146+0000", 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1542999999999998e-05 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2752 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17235968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.229 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9180840139999997, 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015780444000000001, 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2752 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243574+0000", 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243573+0000", 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243576+0000", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.429e-06 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243582+0000", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8499999999999999e-06 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243586+0000", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6489999999999999e-06 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259178+0000", 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015592079999999999 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259255+0000", 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6403999999999996e-05 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259354+0000", 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9415999999999998e-05 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2753 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9180675310000002, 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015953208, 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2753 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243590+0000", 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243592+0000", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8810000000000001e-06 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243597+0000", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6110000000000001e-06 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243602+0000", 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.318e-06 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259382+0000", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01578061 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259455+0000", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2459000000000005e-05 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259543+0000", 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8617999999999998e-05 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2754 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 17891328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9180514500000001, 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016076594999999999, 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2754 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243606+0000", 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243608+0000", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0269999999999998e-06 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243613+0000", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4999999999999999e-06 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243618+0000", 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4490000000000001e-06 2024-09-21T14:21:29.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259579+0000", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015961177 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259624+0000", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5064000000000001e-05 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259683+0000", 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8666999999999999e-05 2024-09-21T14:21:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2755 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18219008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9180357209999999, 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016281739999999999, 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2755 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243622+0000", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243624+0000", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9939999999999999e-06 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243629+0000", 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5759999999999996e-06 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243634+0000", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4270000000000001e-06 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259733+0000", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016098918 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259808+0000", 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5228999999999994e-05 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259904+0000", 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5873999999999997e-05 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:2756 3.9 3:963030d0:::rbd_data.11219861e469.0000000000000002:5 [sparse-read 18546688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9180203640000002, 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016423218, 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:29.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2756 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:29.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243637+0000", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243639+0000", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.973e-06 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243644+0000", 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3249999999999998e-06 2024-09-21T14:21:29.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.243648+0000", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1590000000000003e-06 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.259929+0000", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016280735000000001 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.260004+0000", 2024-09-21T14:21:29.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4855000000000001e-05 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:24.260061+0000", 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6468000000000002e-05 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:29.250 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:29.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:29.252 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:21:29.253 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:29.251+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:29.266 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone4 intact 2024-09-21T14:21:29.352 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:29.352+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:7269 3.19 3:9df50081:::rbd_header.11212f271376:head [call rbd.get_access_timestamp in=23b out=8b,call rbd.get_modify_timestamp in=23b out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.357873+0000", 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.00015292699999999999, 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00017685199999999999, 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.357 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 7269 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357873+0000", 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357873+0000", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357873+0000", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357876+0000", 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.056e-06 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357882+0000", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7880000000000003e-06 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357885+0000", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5470000000000001e-06 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357913+0000", 2024-09-21T14:21:29.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7603999999999999e-05 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.357961+0000", 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8640000000000002e-05 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.358050+0000", 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8722999999999997e-05 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.361 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-09-21T14:21:29.362 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 1 2024-09-21T14:21:29.362 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:29.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:29.376 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T14:21:29.377 INFO:teuthology.orchestra.run.smithi073.stdout:819 write 0x789dd3c thru 0x78a28ba (0x4b7f bytes) 2024-09-21T14:21:29.389 INFO:teuthology.orchestra.run.smithi073.stdout:820 read 0x4fca3b4 thru 0x4fd6d82 (0xc9cf bytes) 2024-09-21T14:21:29.389 INFO:teuthology.orchestra.run.smithi073.stdout:821 write 0x312e675 thru 0x3137302 (0x8c8e bytes) 2024-09-21T14:21:29.424 INFO:teuthology.orchestra.run.smithi073.stdout:822 punch from 0x2841db to 0x285561, (0x1386 bytes) 2024-09-21T14:21:29.424 INFO:teuthology.orchestra.run.smithi073.stdout:824 write 0x782c287 thru 0x78375ef (0xb369 bytes) 2024-09-21T14:21:29.426 INFO:teuthology.orchestra.run.smithi073.stdout:825 punch from 0x2f74fa7 to 0x2f76ae5, (0x1b3e bytes) 2024-09-21T14:21:29.426 INFO:teuthology.orchestra.run.smithi073.stdout:826 read 0x1f6c517 thru 0x1f71cce (0x57b8 bytes) 2024-09-21T14:21:29.426 INFO:teuthology.orchestra.run.smithi073.stdout:828 read 0x3cdba50 thru 0x3cdc7c1 (0xd72 bytes) 2024-09-21T14:21:29.426 INFO:teuthology.orchestra.run.smithi073.stdout:829 write 0x3007249 thru 0x300eb88 (0x7940 bytes) 2024-09-21T14:21:29.453 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:29.453+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:29.454 INFO:teuthology.orchestra.run.smithi073.stdout:831 write 0x5ed551e thru 0x5ed6ee4 (0x19c7 bytes) 2024-09-21T14:21:29.466 INFO:teuthology.orchestra.run.smithi073.stdout:832 writesame 0x339c49f thru 0x33a5ca1 data_size 0x2b(0x9803 bytes) 2024-09-21T14:21:29.515 INFO:teuthology.orchestra.run.smithi073.stdout:833 writesame 0xe1b1226 thru 0xe1b7a26 data_size 0x4b(0x6801 bytes) 2024-09-21T14:21:29.520 INFO:teuthology.orchestra.run.smithi073.stdout:834 trunc from 0xe1b7a27 to 0xa2131fe 2024-09-21T14:21:29.526 INFO:teuthology.orchestra.run.smithi073.stdout:835 writesame 0x6613430 thru 0x661caec data_size 0x13(0x96bd bytes) 2024-09-21T14:21:29.527 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:29.527 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:29.527 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:29.527 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:29.527 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:29.537 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:29.538 INFO:teuthology.orchestra.run.smithi073.stdout:837 read 0x3c3ca13 thru 0x3c4620c (0x97fa bytes) 2024-09-21T14:21:29.538 INFO:teuthology.orchestra.run.smithi073.stdout:840 trunc from 0xa2131fe to 0x9564403 2024-09-21T14:21:29.544 INFO:teuthology.orchestra.run.smithi073.stdout:841 write 0x4a683c0 thru 0x4a702a1 (0x7ee2 bytes) 2024-09-21T14:21:29.552 INFO:teuthology.orchestra.run.smithi073.stdout:842 read 0x202e90d thru 0x2036490 (0x7b84 bytes) 2024-09-21T14:21:29.552 INFO:teuthology.orchestra.run.smithi073.stdout:843 read 0x9174b7 thru 0x921d05 (0xa84f bytes) 2024-09-21T14:21:29.559 INFO:teuthology.orchestra.run.smithi073.stdout:844 write 0x9e5a6ec thru 0x9e5a847 (0x15c bytes) 2024-09-21T14:21:29.563 INFO:teuthology.orchestra.run.smithi073.stdout:846 punch from 0x740984a to 0x7412671, (0x8e27 bytes) 2024-09-21T14:21:29.563 INFO:teuthology.orchestra.run.smithi073.stdout:847 read 0x68f147f thru 0x68f808c (0x6c0e bytes) 2024-09-21T14:21:29.563 INFO:teuthology.orchestra.run.smithi073.stdout:848 write 0x3ee3ca1 thru 0x3ef20cd (0xe42d bytes) 2024-09-21T14:21:29.575 INFO:teuthology.orchestra.run.smithi073.stdout:849 read 0x8b1cab3 thru 0x8b27e53 (0xb3a1 bytes) 2024-09-21T14:21:29.575 INFO:teuthology.orchestra.run.smithi073.stdout:850 read 0x4fe34eb thru 0x4fee753 (0xb269 bytes) 2024-09-21T14:21:29.581 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:29.582+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:29.586 INFO:teuthology.orchestra.run.smithi073.stdout:851 writesame 0xe4d1ac6 thru 0xe4d6bf9 data_size 0x2(0x5134 bytes) 2024-09-21T14:21:29.586 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.586 INFO:teuthology.orchestra.run.smithi073.stdout:853 read 0x95e3bb9 thru 0x95e8a06 (0x4e4e bytes) 2024-09-21T14:21:29.586 INFO:teuthology.orchestra.run.smithi073.stdout:854 write 0x25ebff7 thru 0x25efc1a (0x3c24 bytes) 2024-09-21T14:21:29.631 INFO:teuthology.orchestra.run.smithi073.stdout:855 read 0x72aeb0a thru 0x72ba759 (0xbc50 bytes) 2024-09-21T14:21:29.632 INFO:teuthology.orchestra.run.smithi073.stdout:856 punch from 0xbe746ec to 0xbe7f92f, (0xb243 bytes) 2024-09-21T14:21:29.632 INFO:teuthology.orchestra.run.smithi073.stdout:857 read 0x2dff576 thru 0x2e060f2 (0x6b7d bytes) 2024-09-21T14:21:29.633 INFO:teuthology.orchestra.run.smithi073.stdout:859 writesame 0x523ca55 thru 0x5246116 data_size 0x17(0x96c2 bytes) 2024-09-21T14:21:29.655 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:29.655+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8827 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589183+0000", 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074299978000000003, 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018397700999999999, 2024-09-21T14:21:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8827 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589183+0000", 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589183+0000", 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589183+0000", 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589185+0000", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1519999999999999e-06 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589191+0000", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589196+0000", 2024-09-21T14:21:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8890000000000001e-06 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607422+0000", 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018225771000000002 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607495+0000", 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3193000000000002e-05 2024-09-21T14:21:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607581+0000", 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5903999999999996e-05 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8828 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589200+0000", 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074282848999999998, 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018572143999999999, 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8828 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589200+0000", 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589200+0000", 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589200+0000", 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589202+0000", 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0930000000000001e-06 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589208+0000", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2389999999999999e-06 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589213+0000", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4560000000000002e-06 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607610+0000", 2024-09-21T14:21:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018396688000000001 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607685+0000", 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5077e-05 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607772+0000", 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7897000000000002e-05 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8829 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589217+0000", 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074266102000000001, 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018758851, 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8829 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589217+0000", 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589217+0000", 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589217+0000", 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589219+0000", 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.272e-06 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589225+0000", 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.314e-06 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589230+0000", 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3309999999999997e-06 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607801+0000", 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018571552000000002 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607886+0000", 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4320000000000006e-05 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.607976+0000", 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0395000000000004e-05 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8830 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589234+0000", 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074249258999999998, 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018913528999999998, 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8830 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589234+0000", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589234+0000", 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589234+0000", 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589236+0000", 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2510000000000001e-06 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589242+0000", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0279999999999997e-06 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589246+0000", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4889999999999997e-06 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608003+0000", 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018756591999999999 2024-09-21T14:21:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608079+0000", 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6432000000000003e-05 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608147+0000", 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8122999999999998e-05 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8831 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589264+0000", 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074218866999999994, 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019056600999999999, 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8831 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589264+0000", 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589264+0000", 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589264+0000", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589267+0000", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589274+0000", 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2930000000000003e-06 2024-09-21T14:21:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589279+0000", 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9910000000000002e-06 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608175+0000", 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018896238999999999 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608250+0000", 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5230999999999999e-05 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608321+0000", 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0648000000000005e-05 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8832 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589283+0000", 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074200039999999995, 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019210468000000001, 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8832 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589283+0000", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589283+0000", 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589283+0000", 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589285+0000", 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1639999999999999e-06 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589291+0000", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589296+0000", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8779999999999997e-06 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608351+0000", 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019054469000000001 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608425+0000", 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4592999999999995e-05 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608494+0000", 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8332999999999998e-05 2024-09-21T14:21:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8833 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589301+0000", 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074182660999999997, 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019364131999999999, 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8833 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589301+0000", 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589301+0000", 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589300+0000", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589302+0000", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.356e-06 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589309+0000", 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1580000000000002e-06 2024-09-21T14:21:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589313+0000", 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8729999999999998e-06 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608521+0000", 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019207848999999999 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608594+0000", 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3177999999999996e-05 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608665+0000", 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0188e-05 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8834 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589324+0000", 2024-09-21T14:21:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074158795, 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019514019000000001, 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8834 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589324+0000", 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589324+0000", 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589324+0000", 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589326+0000", 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589333+0000", 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7109999999999998e-06 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589338+0000", 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.848e-06 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608691+0000", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019353696 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608763+0000", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1925e-05 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608838+0000", 2024-09-21T14:21:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.504e-05 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8835 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589342+0000", 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074141307000000004, 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019667289000000001, 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8835 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589342+0000", 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589342+0000", 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589342+0000", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589349+0000", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1550000000000001e-06 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589355+0000", 2024-09-21T14:21:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5860000000000001e-06 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589361+0000", 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608865+0000", 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019504731000000001 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.608938+0000", 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2566999999999998e-05 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609009+0000", 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1339000000000003e-05 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8836 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589365+0000", 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074118289000000004, 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019820935000000001, 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8836 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589365+0000", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589365+0000", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589365+0000", 2024-09-21T14:21:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589367+0000", 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1639999999999999e-06 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589372+0000", 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.711e-06 2024-09-21T14:21:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589377+0000", 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609038+0000", 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019661201 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609115+0000", 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7218999999999994e-05 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609186+0000", 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0388000000000004e-05 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8837 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589383+0000", 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074100470000000002, 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019986233999999999, 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8837 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589383+0000", 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589383+0000", 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589382+0000", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589384+0000", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.086e-06 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589391+0000", 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:21:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589395+0000", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.622e-06 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609219+0000", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019823931999999999 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609293+0000", 2024-09-21T14:21:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3564000000000002e-05 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609369+0000", 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6044999999999994e-05 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8838 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589400+0000", 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074083650000000001, 2024-09-21T14:21:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020135107999999999, 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8838 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589400+0000", 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589400+0000", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589399+0000", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589401+0000", 2024-09-21T14:21:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.956e-06 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589409+0000", 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3930000000000002e-06 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589413+0000", 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.933e-06 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609395+0000", 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019981704999999999 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609469+0000", 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3693000000000001e-05 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609535+0000", 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5777000000000004e-05 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8839 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589418+0000", 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074065338999999994, 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020289959999999999, 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8839 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589418+0000", 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589418+0000", 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589417+0000", 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589420+0000", 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0810000000000001e-06 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589426+0000", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3300000000000004e-06 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589431+0000", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.809e-06 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609563+0000", 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020132351999999999 2024-09-21T14:21:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609637+0000", 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4128000000000006e-05 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609708+0000", 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0642999999999994e-05 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8840 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589435+0000", 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074048038999999996, 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020451496, 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8840 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589435+0000", 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589435+0000", 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589435+0000", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589437+0000", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2800000000000002e-06 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589443+0000", 2024-09-21T14:21:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0769999999999996e-06 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589448+0000", 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.843e-06 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609735+0000", 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020287303 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609811+0000", 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5580999999999993e-05 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609887+0000", 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5760000000000006e-05 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.732 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8841 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589465+0000", 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.074018471000000002, 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020630952000000001, 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8841 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589465+0000", 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589465+0000", 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589464+0000", 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589467+0000", 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.396e-06 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589474+0000", 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.593e-06 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589479+0000", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4610000000000001e-06 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.609921+0000", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020442116 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610018+0000", 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.6946000000000003e-05 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610096+0000", 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7892000000000006e-05 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8842 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589486+0000", 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.073997029000000006, 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020812726, 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.737 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8842 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589486+0000", 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589486+0000", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589486+0000", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589488+0000", 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589493+0000", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589498+0000", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610136+0000", 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020637861 2024-09-21T14:21:29.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610217+0000", 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0677e-05 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610299+0000", 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2367000000000006e-05 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8843 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589502+0000", 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.073981541999999997, 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021498626999999999, 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8843 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.742 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589502+0000", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589502+0000", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589501+0000", 2024-09-21T14:21:29.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589503+0000", 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0200000000000001e-06 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589513+0000", 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7960000000000007e-06 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589518+0000", 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6569999999999996e-06 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610342+0000", 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020824669000000001 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610419+0000", 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6737999999999996e-05 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.611000+0000", 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00058110300000000002 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8844 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589522+0000", 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.073961051999999999, 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021136756, 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.746 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8844 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589522+0000", 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589522+0000", 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589522+0000", 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589524+0000", 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1840000000000002e-06 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589530+0000", 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589534+0000", 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4719999999999997e-06 2024-09-21T14:21:29.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610510+0000", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020975434000000001 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610591+0000", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1110000000000001e-05 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610659+0000", 2024-09-21T14:21:29.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7848999999999994e-05 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8845 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589538+0000", 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.073944781000000001, 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021296846000000001, 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8845 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589538+0000", 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589538+0000", 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589538+0000", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589542+0000", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7720000000000001e-06 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589547+0000", 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6019999999999999e-06 2024-09-21T14:21:29.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589553+0000", 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3419999999999998e-06 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610686+0000", 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021133116 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610760+0000", 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4012999999999998e-05 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610835+0000", 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5345000000000005e-05 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8846 3.7 3:e04e9248:::rbd_data.11214feba615.0000000000000046:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.589557+0000", 2024-09-21T14:21:29.755 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.073926504000000004, 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021437947999999998, 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8846 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589557+0000", 2024-09-21T14:21:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589557+0000", 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589556+0000", 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:29.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589558+0000", 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1179999999999999e-06 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589564+0000", 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.609e-06 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.589568+0000", 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4290000000000003e-06 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610858+0000", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.021289731999999999 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610930+0000", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1600000000000006e-05 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.610995+0000", 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4826999999999994e-05 2024-09-21T14:21:29.759 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.760 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.760 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.760 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:29.760 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:29.760 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:29.761 INFO:teuthology.orchestra.run.smithi073.stdout:864 trunc from 0xe4d6bfa to 0xa16b76 2024-09-21T14:21:29.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:29.762 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:29.761+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:29.771 INFO:teuthology.orchestra.run.smithi073.stdout:866 trunc from 0xa16b76 to 0x1d7da89 2024-09-21T14:21:29.777 INFO:teuthology.orchestra.run.smithi073.stdout:870 read 0x8c5819 thru 0x8d29fb (0xd1e3 bytes) 2024-09-21T14:21:29.778 INFO:teuthology.orchestra.run.smithi073.stdout:871 read 0x1bed7ed thru 0x1bf3647 (0x5e5b bytes) 2024-09-21T14:21:29.778 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.785 INFO:teuthology.orchestra.run.smithi073.stdout:873 trunc from 0x1d7da89 to 0x7eeb7ac 2024-09-21T14:21:29.791 INFO:teuthology.orchestra.run.smithi073.stdout:874 read 0x4ff631e thru 0x5000ae2 (0xa7c5 bytes) 2024-09-21T14:21:29.791 INFO:teuthology.orchestra.run.smithi073.stdout:875 write 0x1c8fe56 thru 0x1c96a8c (0x6c37 bytes) 2024-09-21T14:21:29.802 INFO:teuthology.orchestra.run.smithi073.stdout:877 write 0xc7c15d thru 0xc846ee (0x8592 bytes) 2024-09-21T14:21:29.810 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.810 INFO:teuthology.orchestra.run.smithi073.stdout:879 write 0x358de81 thru 0x359208e (0x420e bytes) 2024-09-21T14:21:29.828 INFO:teuthology.orchestra.run.smithi073.stdout:880 trunc from 0x7eeb7ac to 0xeaaa47d 2024-09-21T14:21:29.833 INFO:teuthology.orchestra.run.smithi073.stdout:881 punch from 0x1b04274 to 0x1b081fc, (0x3f88 bytes) 2024-09-21T14:21:29.834 INFO:teuthology.orchestra.run.smithi073.stdout:883 punch from 0x7192f68 to 0x71951f8, (0x2290 bytes) 2024-09-21T14:21:29.834 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.834 INFO:teuthology.orchestra.run.smithi073.stdout:886 read 0x67a050c thru 0x67a59c0 (0x54b5 bytes) 2024-09-21T14:21:29.834 INFO:teuthology.orchestra.run.smithi073.stdout:887 trunc from 0xeaaa47d to 0xe265061 2024-09-21T14:21:29.839 INFO:teuthology.orchestra.run.smithi073.stdout:889 read 0x64e6218 thru 0x64f4407 (0xe1f0 bytes) 2024-09-21T14:21:29.840 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.840 INFO:teuthology.orchestra.run.smithi073.stdout:891 read 0xa150c33 thru 0xa15aeec (0xa2ba bytes) 2024-09-21T14:21:29.840 INFO:teuthology.orchestra.run.smithi073.stdout:892 read 0xa095a12 thru 0xa097b5a (0x2149 bytes) 2024-09-21T14:21:29.845 INFO:teuthology.orchestra.run.smithi073.stdout:893 writesame 0xeacf055 thru 0xeade800 data_size 0x2c(0xf7ac bytes) 2024-09-21T14:21:29.845 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.845 INFO:teuthology.orchestra.run.smithi073.stdout:895 read 0xd0024ed thru 0xd00c159 (0x9c6d bytes) 2024-09-21T14:21:29.845 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:29.857 INFO:teuthology.orchestra.run.smithi073.stdout:898 trunc from 0xeade801 to 0x4de36a5 2024-09-21T14:21:29.862 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:29.862+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:29.874 INFO:teuthology.orchestra.run.smithi073.stdout:899 write 0x9b16e48 thru 0x9b1cc8b (0x5e44 bytes) 2024-09-21T14:21:29.884 INFO:teuthology.orchestra.run.smithi073.stdout:900 clone 6 order 22 su 131072 sc 9 2024-09-21T14:21:29.910 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:29.911 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:29.911 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:29.911 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:29.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:29.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:29.963+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:30.061 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:30.061 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:30.061 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:30.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:30.061 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:30.063 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:30.064+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:30.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:30.165 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:30.165+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46256860799999999, 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:21:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:21:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46255729800000001, 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:21:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46254192199999999, 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:21:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46252439400000001, 2024-09-21T14:21:30.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:21:30.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46250544300000002, 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:21:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:21:30.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.196 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46248906299999998, 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:21:30.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:21:30.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46247983500000001, 2024-09-21T14:21:30.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:21:30.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:21:30.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46246843900000001, 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.206 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:21:30.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46244571200000001, 2024-09-21T14:21:30.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:30.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:21:30.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.215 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46242945600000002, 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:21:30.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:21:30.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46241470099999998, 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:21:30.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:21:30.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:21:30.224 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46240564699999998, 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:21:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:21:30.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46239460300000002, 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:21:30.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:21:30.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46237920199999999, 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:21:30.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.238 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46236250400000001, 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:30.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:21:30.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:21:30.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.242 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46234689699999998, 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:21:30.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.462329775, 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:21:30.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:21:30.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:21:30.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.462311164, 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.251 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:21:30.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.46229183400000001, 2024-09-21T14:21:30.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.259 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.462272197, 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:30.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:30.261 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:21:30.263 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.264 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:30.264 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:30.264 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:30.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:30.265 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:30.276 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone5 from 0x9b1cc8c (overlap 0xa16b76) to 0x8122c4 2024-09-21T14:21:30.307 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:30.307+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:30.367 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:30.367+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:30.415 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:30.415 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:30.419 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:30.419 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:30.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:30.437 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:30.456 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:30.456 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:30.464 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T14:21:30.468 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:30.468+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:30.474 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:30.474 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:30.492 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:30.492 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:30.527 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:30.527 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:30.547 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:30.548 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:30.548 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:30.548 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:30.548 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:30.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:30.562 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:30.562 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:30.570 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:30.570+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:30.589 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:30.589 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:30.669 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:30.669+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:30.669 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:30.669 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:30.679 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:30.732 INFO:teuthology.orchestra.run.smithi073.stdout:902 trunc from 0x9b1cc8c to 0x8668841 2024-09-21T14:21:30.756 INFO:teuthology.orchestra.run.smithi073.stdout:903 write 0x2b988aa thru 0x2b9c59c (0x3cf3 bytes) 2024-09-21T14:21:30.759 INFO:teuthology.orchestra.run.smithi073.stdout:905 read 0x53e3cb0 thru 0x53e9662 (0x59b3 bytes) 2024-09-21T14:21:30.764 INFO:teuthology.orchestra.run.smithi073.stdout:906 write 0x8978999 thru 0x89855c1 (0xcc29 bytes) 2024-09-21T14:21:30.766 INFO:teuthology.orchestra.run.smithi073.stdout:907 write 0x8024c3b thru 0x802e3fb (0x97c1 bytes) 2024-09-21T14:21:30.767 INFO:teuthology.orchestra.run.smithi073.stdout:908 read 0x7730420 thru 0x7730a49 (0x62a bytes) 2024-09-21T14:21:30.767 INFO:teuthology.orchestra.run.smithi073.stdout:909 punch from 0x24be8d to 0x2586d3, (0xc846 bytes) 2024-09-21T14:21:30.768 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:30.768 INFO:teuthology.orchestra.run.smithi073.stdout:912 read 0x1dc60bb thru 0x1dd210d (0xc053 bytes) 2024-09-21T14:21:30.768 INFO:teuthology.orchestra.run.smithi073.stdout:913 write 0x2e0afea thru 0x2e10756 (0x576d bytes) 2024-09-21T14:21:30.770 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:30.770+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:30.771 INFO:teuthology.orchestra.run.smithi073.stdout:915 read 0x1ccbcdb thru 0x1ccdd38 (0x205e bytes) 2024-09-21T14:21:30.772 INFO:teuthology.orchestra.run.smithi073.stdout:918 punch from 0x7b30748 to 0x7b318e2, (0x119a bytes) 2024-09-21T14:21:30.780 INFO:teuthology.orchestra.run.smithi073.stdout:919 writesame 0xb8daad1 thru 0xb8de940 data_size 0x18(0x3e70 bytes) 2024-09-21T14:21:30.780 INFO:teuthology.orchestra.run.smithi073.stdout:921 write 0xa3649d5 thru 0xa373968 (0xef94 bytes) 2024-09-21T14:21:30.783 INFO:teuthology.orchestra.run.smithi073.stdout:923 read 0x71ad412 thru 0x71bb43c (0xe02b bytes) 2024-09-21T14:21:30.796 INFO:teuthology.orchestra.run.smithi073.stdout:924 trunc from 0xb8de941 to 0x1496cbf 2024-09-21T14:21:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:30.851 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:30.893 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:30.893+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:30.970 INFO:teuthology.orchestra.run.smithi073.stdout:925 read 0x1201752 thru 0x1210c71 (0xf520 bytes) 2024-09-21T14:21:30.970 INFO:teuthology.orchestra.run.smithi073.stdout:926 read 0x142b985 thru 0x1434fe8 (0x9664 bytes) 2024-09-21T14:21:30.972 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:30.972+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:30.972 INFO:teuthology.orchestra.run.smithi073.stdout:927 trunc from 0x1496cbf to 0x2ac070c 2024-09-21T14:21:30.989 INFO:teuthology.orchestra.run.smithi073.stdout:928 writesame 0x6ff368b thru 0x7000d75 data_size 0x1(0xd6eb bytes) 2024-09-21T14:21:30.995 INFO:teuthology.orchestra.run.smithi073.stdout:929 trunc from 0x7000d76 to 0xbee41f8 2024-09-21T14:21:31.002 INFO:teuthology.orchestra.run.smithi073.stdout:931 writesame 0x715047 thru 0x715df1 data_size 0x1(0xdab bytes) 2024-09-21T14:21:31.004 INFO:teuthology.orchestra.run.smithi073.stdout:933 punch from 0x3d08b29 to 0x3d09b9b, (0x1072 bytes) 2024-09-21T14:21:31.004 INFO:teuthology.orchestra.run.smithi073.stdout:934 read 0x569262f thru 0x56972f9 (0x4ccb bytes) 2024-09-21T14:21:31.004 INFO:teuthology.orchestra.run.smithi073.stdout:935 write 0xa422130 thru 0xa42b851 (0x9722 bytes) 2024-09-21T14:21:31.011 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.011 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.013 INFO:teuthology.orchestra.run.smithi073.stdout:936 writesame 0xd9307c8 thru 0xd9389d5 data_size 0xb3(0x820e bytes) 2024-09-21T14:21:31.013 INFO:teuthology.orchestra.run.smithi073.stdout:937 writesame 0xc41dc5e thru 0xc427b3b data_size 0x31(0x9ede bytes) 2024-09-21T14:21:31.020 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.020 INFO:teuthology.orchestra.run.smithi073.stdout:939 write 0x6e1f7ba thru 0x6e29587 (0x9dce bytes) 2024-09-21T14:21:31.021 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:31.024 INFO:teuthology.orchestra.run.smithi073.stdout:940 read 0x8aa9156 thru 0x8ab0e65 (0x7d10 bytes) 2024-09-21T14:21:31.024 INFO:teuthology.orchestra.run.smithi073.stdout:941 writesame 0xb49f526 thru 0xb4addfe data_size 0x1(0xe8d9 bytes) 2024-09-21T14:21:31.054 INFO:teuthology.orchestra.run.smithi073.stdout:942 read 0x60e1af6 thru 0x60f0649 (0xeb54 bytes) 2024-09-21T14:21:31.054 INFO:teuthology.orchestra.run.smithi073.stdout:943 writesame 0x3040e65 thru 0x304c24c data_size 0x26(0xb3e8 bytes) 2024-09-21T14:21:31.057 INFO:teuthology.orchestra.run.smithi073.stdout:944 punch from 0x6c44cc3 to 0x6c4988f, (0x4bcc bytes) 2024-09-21T14:21:31.057 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.071 INFO:teuthology.orchestra.run.smithi073.stdout:946 trunc from 0xd9389d6 to 0x1c98ff7 2024-09-21T14:21:31.072 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:31.072+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:31.087 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.088 INFO:teuthology.orchestra.run.smithi073.stdout:948 read 0x13e30bb thru 0x13ea7a9 (0x76ef bytes) 2024-09-21T14:21:31.088 INFO:teuthology.orchestra.run.smithi073.stdout:949 read 0x146bcf3 thru 0x1479617 (0xd925 bytes) 2024-09-21T14:21:31.088 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.093 INFO:teuthology.orchestra.run.smithi073.stdout:951 trunc from 0x1c98ff7 to 0x63e4e8c 2024-09-21T14:21:31.099 INFO:teuthology.orchestra.run.smithi073.stdout:952 punch from 0x1358b4e to 0x1358c97, (0x149 bytes) 2024-09-21T14:21:31.099 INFO:teuthology.orchestra.run.smithi073.stdout:953 read 0x38c78a2 thru 0x38cbe1d (0x457c bytes) 2024-09-21T14:21:31.099 INFO:teuthology.orchestra.run.smithi073.stdout:954 read 0x35d98b3 thru 0x35de020 (0x476e bytes) 2024-09-21T14:21:31.099 INFO:teuthology.orchestra.run.smithi073.stdout:955 read 0x52df4c9 thru 0x52eeb44 (0xf67c bytes) 2024-09-21T14:21:31.099 INFO:teuthology.orchestra.run.smithi073.stdout:956 write 0x5bde2bc thru 0x5bec5ad (0xe2f2 bytes) 2024-09-21T14:21:31.102 INFO:teuthology.orchestra.run.smithi073.stdout:957 punch from 0x3f4aa28 to 0x3f50ded, (0x63c5 bytes) 2024-09-21T14:21:31.103 INFO:teuthology.orchestra.run.smithi073.stdout:959 writesame 0xcc9312 thru 0xcd7285 data_size 0x7e(0xdf74 bytes) 2024-09-21T14:21:31.108 INFO:teuthology.orchestra.run.smithi073.stdout:960 write 0x6620b9b thru 0x6630592 (0xf9f8 bytes) 2024-09-21T14:21:31.117 INFO:teuthology.orchestra.run.smithi073.stdout:962 write 0x6d9ddea thru 0x6da2858 (0x4a6f bytes) 2024-09-21T14:21:31.120 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.120 INFO:teuthology.orchestra.run.smithi073.stdout:964 punch from 0x22ba0b2 to 0x22c69cc, (0xc91a bytes) 2024-09-21T14:21:31.120 INFO:teuthology.orchestra.run.smithi073.stdout:966 read 0x52276d4 thru 0x5231384 (0x9cb1 bytes) 2024-09-21T14:21:31.120 INFO:teuthology.orchestra.run.smithi073.stdout:967 write 0x30015f5 thru 0x300c560 (0xaf6c bytes) 2024-09-21T14:21:31.123 INFO:teuthology.orchestra.run.smithi073.stdout:968 read 0x3ed097e thru 0x3edcbbd (0xc240 bytes) 2024-09-21T14:21:31.128 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.128 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.131 INFO:teuthology.orchestra.run.smithi073.stdout:970 trunc from 0x6da2859 to 0xd71d388 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:972 read 0x30471ba thru 0x304ff7b (0x8dc2 bytes) 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:973 punch from 0x8454651 to 0x8458dd6, (0x4785 bytes) 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:975 read 0x49b53a1 thru 0x49b9241 (0x3ea1 bytes) 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:976 read 0x306df1b thru 0x30773dd (0x94c3 bytes) 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.136 INFO:teuthology.orchestra.run.smithi073.stdout:981 write 0x7e7e57a thru 0x7e7f6c7 (0x114e bytes) 2024-09-21T14:21:31.137 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:31.138 INFO:teuthology.orchestra.run.smithi073.stdout:983 read 0x9f63d9b thru 0x9f701a6 (0xc40c bytes) 2024-09-21T14:21:31.138 INFO:teuthology.orchestra.run.smithi073.stdout:985 writesame 0x505f34a thru 0x506606f data_size 0x3(0x6d26 bytes) 2024-09-21T14:21:31.142 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.142 INFO:teuthology.orchestra.run.smithi073.stdout:988 punch from 0x9637e74 to 0x963aeac, (0x3038 bytes) 2024-09-21T14:21:31.153 INFO:teuthology.orchestra.run.smithi073.stdout:989 trunc from 0xd71d388 to 0x4551821 2024-09-21T14:21:31.166 INFO:teuthology.orchestra.run.smithi073.stdout:990 read 0x3620568 thru 0x3629051 (0x8aea bytes) 2024-09-21T14:21:31.166 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.166 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.173 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:31.173+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:31.177 INFO:teuthology.orchestra.run.smithi073.stdout:993 write 0xa543f0e thru 0xa553498 (0xf58b bytes) 2024-09-21T14:21:31.181 INFO:teuthology.orchestra.run.smithi073.stdout:994 write 0x513812c thru 0x513b60c (0x34e1 bytes) 2024-09-21T14:21:31.183 INFO:teuthology.orchestra.run.smithi073.stdout:995 writesame 0x69ba809 thru 0x69c1ba1 data_size 0x1(0x7399 bytes) 2024-09-21T14:21:31.197 INFO:teuthology.orchestra.run.smithi073.stdout:996 trunc from 0xa553499 to 0xb7473b7 2024-09-21T14:21:31.203 INFO:teuthology.orchestra.run.smithi073.stdout:997 punch from 0xb0ccdeb to 0xb0ce635, (0x184a bytes) 2024-09-21T14:21:31.203 INFO:teuthology.orchestra.run.smithi073.stdout:998 read 0x204a00b thru 0x204feba (0x5eb0 bytes) 2024-09-21T14:21:31.203 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.203 INFO:teuthology.orchestra.run.smithi073.stdout:1005 writesame 0x1d3bb71 thru 0x1d3f56a data_size 0x2(0x39fa bytes) 2024-09-21T14:21:31.207 INFO:teuthology.orchestra.run.smithi073.stdout:1007 write 0x793113e thru 0x7934ed6 (0x3d99 bytes) 2024-09-21T14:21:31.209 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.210 INFO:teuthology.orchestra.run.smithi073.stdout:1010 punch from 0x10d4561 to 0x10d4b01, (0x5a0 bytes) 2024-09-21T14:21:31.210 INFO:teuthology.orchestra.run.smithi073.stdout:1011 write 0x79f432b thru 0x7a01610 (0xd2e6 bytes) 2024-09-21T14:21:31.213 INFO:teuthology.orchestra.run.smithi073.stdout:1012 write 0x458d0dc thru 0x459aaef (0xda14 bytes) 2024-09-21T14:21:31.216 INFO:teuthology.orchestra.run.smithi073.stdout:1018 read 0x8e02601 thru 0x8e108e9 (0xe2e9 bytes) 2024-09-21T14:21:31.216 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.216 INFO:teuthology.orchestra.run.smithi073.stdout:1022 punch from 0x7f4b780 to 0x7f580ff, (0xc97f bytes) 2024-09-21T14:21:31.216 INFO:teuthology.orchestra.run.smithi073.stdout:1025 read 0x7e66647 thru 0x7e6f6c1 (0x907b bytes) 2024-09-21T14:21:31.228 INFO:teuthology.orchestra.run.smithi073.stdout:1026 trunc from 0xb7473b7 to 0x1767d50 2024-09-21T14:21:31.240 INFO:teuthology.orchestra.run.smithi073.stdout:1028 punch from 0xa26a3e to 0xa30dcc, (0xa38e bytes) 2024-09-21T14:21:31.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:1031 writesame 0x89a65ee thru 0x89a89dc data_size 0x1(0x23ef bytes) 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:1032 read 0x4d84a4a thru 0x4d8eb62 (0xa119 bytes) 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:1039 read 0x4349153 thru 0x4349ef9 (0xda7 bytes) 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:1040 punch from 0x35f57da to 0x35ff465, (0x9c8b bytes) 2024-09-21T14:21:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:1041 write 0x4ee9742 thru 0x4ef5697 (0xbf56 bytes) 2024-09-21T14:21:31.257 INFO:teuthology.orchestra.run.smithi073.stdout:1042 read 0x2336351 thru 0x234332e (0xcfde bytes) 2024-09-21T14:21:31.257 INFO:teuthology.orchestra.run.smithi073.stdout:1043 read 0x2fb5899 thru 0x2fc1a20 (0xc188 bytes) 2024-09-21T14:21:31.257 INFO:teuthology.orchestra.run.smithi073.stdout:1044 read 0x56c89be thru 0x56d355d (0xaba0 bytes) 2024-09-21T14:21:31.258 INFO:teuthology.orchestra.run.smithi073.stdout:1045 read 0x7a54722 thru 0x7a5ebd4 (0xa4b3 bytes) 2024-09-21T14:21:31.258 INFO:teuthology.orchestra.run.smithi073.stdout:1046 writesame 0x5d0d9d8 thru 0x5d1cdde data_size 0x1(0xf407 bytes) 2024-09-21T14:21:31.260 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.260 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:31.274 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:31.274+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:31.293 INFO:teuthology.orchestra.run.smithi073.stdout:1047 write 0xa54171e thru 0xa54f359 (0xdc3c bytes) 2024-09-21T14:21:31.306 INFO:teuthology.orchestra.run.smithi073.stdout:1050 write 0xdac14e9 thru 0xdacd7f8 (0xc310 bytes) 2024-09-21T14:21:31.323 INFO:teuthology.orchestra.run.smithi073.stdout:1051 trunc from 0xdacd7f9 to 0x37c8d06 2024-09-21T14:21:31.345 INFO:teuthology.orchestra.run.smithi073.stdout:1052 writesame 0x6f4a374 thru 0x6f51805 data_size 0x2b(0x7492 bytes) 2024-09-21T14:21:31.345 INFO:teuthology.orchestra.run.smithi073.stdout:1054 read 0x158cdc6 thru 0x1591aa7 (0x4ce2 bytes) 2024-09-21T14:21:31.350 INFO:teuthology.orchestra.run.smithi073.stdout:1055 writesame 0x74b80f9 thru 0x74bc8e8 data_size 0x8(0x47f0 bytes) 2024-09-21T14:21:31.358 INFO:teuthology.orchestra.run.smithi073.stdout:1056 write 0x9476445 thru 0x94840ef (0xdcab bytes) 2024-09-21T14:21:31.362 INFO:teuthology.orchestra.run.smithi073.stdout:1058 read 0x2c42174 thru 0x2c456f8 (0x3585 bytes) 2024-09-21T14:21:31.362 INFO:teuthology.orchestra.run.smithi073.stdout:1060 read 0x2c86aba thru 0x2c9199d (0xaee4 bytes) 2024-09-21T14:21:31.362 INFO:teuthology.orchestra.run.smithi073.stdout:1061 writesame 0x12879de thru 0x12959ed data_size 0x78(0xe010 bytes) 2024-09-21T14:21:31.364 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.371 INFO:teuthology.orchestra.run.smithi073.stdout:1063 writesame 0xab60a6e thru 0xab67fd4 data_size 0x1(0x7567 bytes) 2024-09-21T14:21:31.371 INFO:teuthology.orchestra.run.smithi073.stdout:1064 writesame 0x9820308 thru 0x982f00f data_size 0xa4(0xed08 bytes) 2024-09-21T14:21:31.374 INFO:teuthology.orchestra.run.smithi073.stdout:1066 writesame 0x3c72774 thru 0x3c74f48 data_size 0xb(0x27d5 bytes) 2024-09-21T14:21:31.375 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:31.375+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:31.377 INFO:teuthology.orchestra.run.smithi073.stdout:1067 punch from 0x49b2ea2 to 0x49b49c2, (0x1b20 bytes) 2024-09-21T14:21:31.385 INFO:teuthology.orchestra.run.smithi073.stdout:1069 write 0xcdc6433 thru 0xcdd5e1f (0xf9ed bytes) 2024-09-21T14:21:31.389 INFO:teuthology.orchestra.run.smithi073.stdout:1070 writesame 0x9e53e58 thru 0x9e5e33b data_size 0x4(0xa4e4 bytes) 2024-09-21T14:21:31.396 INFO:teuthology.orchestra.run.smithi073.stdout:1071 read 0x591443f thru 0x5918866 (0x4428 bytes) 2024-09-21T14:21:31.396 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:31.396 INFO:teuthology.orchestra.run.smithi073.stdout:1074 read 0x5c7d9e1 thru 0x5c7ec53 (0x1273 bytes) 2024-09-21T14:21:31.396 INFO:teuthology.orchestra.run.smithi073.stdout:1075 write 0x2c8dda3 thru 0x2c9d4f1 (0xf74f bytes) 2024-09-21T14:21:31.399 INFO:teuthology.orchestra.run.smithi073.stdout:1076 writesame 0x7c87164 thru 0x7c8f247 data_size 0x49(0x80e4 bytes) 2024-09-21T14:21:31.401 INFO:teuthology.orchestra.run.smithi073.stdout:1078 read 0x392719e thru 0x3929b5a (0x29bd bytes) 2024-09-21T14:21:31.402 INFO:teuthology.orchestra.run.smithi073.stdout:1082 read 0x2220f0 thru 0x22d2c0 (0xb1d1 bytes) 2024-09-21T14:21:31.402 INFO:teuthology.orchestra.run.smithi073.stdout:1084 clone 7 order 20 su 65536 sc 4 2024-09-21T14:21:31.418 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.418 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:31.512 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:31.512+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:31.571 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.571 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.577 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:31.577+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:31.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:31.677 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:31.677+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:31.723 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.723 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:31.778 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:31.778+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:31.873 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:31.873 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:31.879 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:31.879+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:31.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:31.980 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:31.980+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.032 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:32.081 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:32.081+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:32.135 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.136 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:32.151 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:32.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:21:32.151 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T14:21:32.151 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.754545 2024-09-21T14:21:32.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.7545454855377425 2024-09-21T14:21:32.188 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:32.188+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:32.205 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:21:32.254 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.254 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:32.300 INFO:teuthology.orchestra.run.smithi073.stdout:flattening image image_client.0-clone6 2024-09-21T14:21:32.300 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:32.300+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:32.384 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:32.384+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:32.410 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.410 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:32.474 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T14:21:32.485 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:32.485+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:32.586 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:32.586+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:32.597 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.597 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:32.606 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:32.687 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:32.687+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:32.725 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:32.726 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:32.746 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:32.746 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:32.758 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:32.758 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:32.762 INFO:teuthology.orchestra.run.smithi073.stdout:1085 read 0x71a4a46 thru 0x71b20c9 (0xd684 bytes) 2024-09-21T14:21:32.763 INFO:teuthology.orchestra.run.smithi073.stdout:1086 punch from 0x85b9d45 to 0x85b9de6, (0xa1 bytes) 2024-09-21T14:21:32.767 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:32.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:32.769 INFO:teuthology.orchestra.run.smithi073.stdout:1089 clone 8 order 20 su 32768 sc 2 2024-09-21T14:21:32.783 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:32.788 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:32.788+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:32.800 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:32.800 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:32.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:32.833 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:32.849 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:32.850 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:32.875 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:32.876 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 23.332572247000002, 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:32.877 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:32.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:32.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:32.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9347 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2775676890000001, 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031239744999999999, 2024-09-21T14:21:32.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9347 2024-09-21T14:21:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596945+0000", 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7199999999999997e-07 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596947+0000", 2024-09-21T14:21:32.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7609999999999999e-06 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596950+0000", 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.604e-06 2024-09-21T14:21:32.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628112+0000", 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031161525999999998 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628148+0000", 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6531e-05 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628184+0000", 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5559999999999998e-05 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9348 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2775596249999999, 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031324221999999999, 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.901 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9348 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.902 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596953+0000", 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6299999999999993e-07 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596956+0000", 2024-09-21T14:21:32.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9950000000000001e-06 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596961+0000", 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.826e-06 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628204+0000", 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031243602999999998 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628240+0000", 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5958999999999999e-05 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628276+0000", 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.608e-05 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9349 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2775474899999999, 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031404135, 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.911 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9349 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.912 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596966+0000", 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2859999999999998e-06 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596972+0000", 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:21:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596977+0000", 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6090000000000003e-06 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628296+0000", 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031319461999999999 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628332+0000", 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6266000000000001e-05 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628368+0000", 2024-09-21T14:21:32.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5994999999999997e-05 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9350 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.920 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2775311399999998, 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031481451000000001, 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9350 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:32.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:32.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596982+0000", 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4300000000000001e-06 2024-09-21T14:21:32.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596992+0000", 2024-09-21T14:21:32.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0679999999999999e-05 2024-09-21T14:21:32.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596995+0000", 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6070000000000002e-06 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628389+0000", 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031393873000000003 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628426+0000", 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7013e-05 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628462+0000", 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6117e-05 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9351 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:32.928 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.277514279, 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031558256999999999, 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9351 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596998+0000", 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3200000000000001e-06 2024-09-21T14:21:32.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597001+0000", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9129999999999999e-06 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597004+0000", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.435e-06 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628483+0000", 2024-09-21T14:21:32.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031478902000000003 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628520+0000", 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7670999999999999e-05 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628556+0000", 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5247000000000003e-05 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9352 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2775057699999999, 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031641458999999997, 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9352 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597012+0000", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4790000000000001e-06 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597015+0000", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.174e-06 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597019+0000", 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.461e-06 2024-09-21T14:21:32.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628575+0000", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031556661 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628611+0000", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5345999999999997e-05 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628647+0000", 2024-09-21T14:21:32.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6548999999999999e-05 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9353 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.277485499, 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031734509000000001, 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9353 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:32.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597028+0000", 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597034+0000", 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9039999999999997e-06 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597038+0000", 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.639e-06 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628676+0000", 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031637327 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628725+0000", 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9208e-05 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628761+0000", 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5608000000000002e-05 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9354 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.941 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.277468711, 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031811304999999998, 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9354 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:32.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597042+0000", 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597045+0000", 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3999999999999999e-06 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597050+0000", 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4310000000000002e-06 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597054+0000", 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.489e-06 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628780+0000", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031726733999999999 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628817+0000", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6525e-05 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628854+0000", 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7283000000000002e-05 2024-09-21T14:21:32.945 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9355 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2774542979999999, 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031891615999999998, 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.946 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9355 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597059+0000", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.854e-06 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597064+0000", 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0019999999999997e-06 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597069+0000", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1429999999999997e-06 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628874+0000", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031805319999999998 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628912+0000", 2024-09-21T14:21:32.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8102000000000002e-05 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628949+0000", 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6514999999999999e-05 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9356 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2774384840000002, 2024-09-21T14:21:32.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031979298000000003, 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9356 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597075+0000", 2024-09-21T14:21:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1210000000000001e-06 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597082+0000", 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.024e-06 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597086+0000", 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628970+0000", 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031883504 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629014+0000", 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4458e-05 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629052+0000", 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7996e-05 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.954 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9357 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2774210240000001, 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032057078000000003, 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9357 2024-09-21T14:21:32.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597090+0000", 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597098+0000", 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1489999999999994e-06 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597106+0000", 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3769999999999998e-06 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597110+0000", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6709999999999998e-06 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629074+0000", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031963250999999998 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629111+0000", 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7855000000000003e-05 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629148+0000", 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6171999999999999e-05 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9358 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.2773965949999999, 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032116296000000003, 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9358 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597117+0000", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1919999999999999e-06 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597123+0000", 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5359999999999998e-06 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597128+0000", 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3460000000000004e-06 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629161+0000", 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032033211999999998 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629197+0000", 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5896999999999999e-05 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629231+0000", 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4530999999999999e-05 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10578 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.963 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1652128859999999, 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031326523000000002, 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10578 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:32.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709298+0000", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709300+0000", 2024-09-21T14:21:32.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6869999999999999e-06 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709305+0000", 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0289999999999996e-06 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709309+0000", 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.16e-06 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740437+0000", 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031127598999999999 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740525+0000", 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8591000000000006e-05 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740625+0000", 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9724999999999998e-05 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10580 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1651984789999998, 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031423999000000001, 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10580 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709315+0000", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0059999999999999e-06 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709320+0000", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3499999999999996e-06 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709325+0000", 2024-09-21T14:21:32.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4190000000000004e-06 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740638+0000", 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031313682000000002 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740687+0000", 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9196999999999997e-05 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740737+0000", 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9675000000000001e-05 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10585 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1651844759999999, 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031620403999999998, 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10585 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709328+0000", 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709331+0000", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.954e-06 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709333+0000", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2570000000000002e-06 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740768+0000", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031434877999999999 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740855+0000", 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7128999999999999e-05 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740948+0000", 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2039000000000004e-05 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10587 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1651760449999999, 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031726081000000003, 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10587 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709335+0000", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:32.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0899999999999999e-06 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709339+0000", 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7760000000000002e-06 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709342+0000", 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.351e-06 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740960+0000", 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031618865000000003 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741012+0000", 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1249999999999999e-05 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741062+0000", 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9950000000000001e-05 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10591 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.165166556, 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031949914000000003, 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10591 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.981 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:32.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709349+0000", 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4710000000000001e-06 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709352+0000", 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1470000000000001e-06 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709356+0000", 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0639999999999998e-06 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741097+0000", 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031741129 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741185+0000", 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8671000000000005e-05 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741295+0000", 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109634 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10593 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1651537589999998, 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032050040000000002, 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.985 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10593 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709359+0000", 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.232e-06 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709362+0000", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7580000000000001e-06 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709367+0000", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9220000000000002e-06 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741319+0000", 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031951949 2024-09-21T14:21:32.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741370+0000", 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.075e-05 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741408+0000", 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7667999999999999e-05 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10596 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.1651147900000001, 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032199647999999997, 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10596 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709396+0000", 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709398+0000", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7150000000000001e-06 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709402+0000", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3500000000000001e-06 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709404+0000", 2024-09-21T14:21:32.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4329999999999998e-06 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741437+0000", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032033461999999999 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741516+0000", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8345999999999998e-05 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741596+0000", 2024-09-21T14:21:32.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0650999999999997e-05 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:32.994 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:32.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:32.996 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:32.995+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:33.098 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:33.098+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:33.099 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.099 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:33.099 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:33.099 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:33.199 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:33.199+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.3 primary-affinity to 0.754545 (8494492) 2024-09-21T14:21:33.252 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:33.252 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.261 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:33.300 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:33.300+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:33.363 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.363 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:33.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:33.363 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10526 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 466944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.726901+0000", 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6361192230000001, 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033168820000000002, 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10526 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.364 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726901+0000", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726901+0000", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726900+0000", 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726902+0000", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7659999999999999e-06 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726905+0000", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.219e-06 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726908+0000", 2024-09-21T14:21:33.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.424e-06 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.759952+0000", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033044395999999997 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760003+0000", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1310000000000002e-05 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760070+0000", 2024-09-21T14:21:33.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6104000000000003e-05 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10531 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.726921+0000", 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6360985459999999, 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033276039, 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.368 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10531 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726921+0000", 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726921+0000", 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726921+0000", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726923+0000", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7829999999999999e-06 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726926+0000", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0599999999999999e-06 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.726929+0000", 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1700000000000001e-06 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760101+0000", 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033172277 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.371 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760158+0000", 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6486000000000001e-05 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760197+0000", 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9672999999999998e-05 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10543 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 18432~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.372 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727031+0000", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6359888680000001, 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033353657000000002, 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10543 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727031+0000", 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727031+0000", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727031+0000", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727033+0000", 2024-09-21T14:21:33.374 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727039+0000", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.891e-06 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727044+0000", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8199999999999996e-06 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760225+0000", 2024-09-21T14:21:33.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033181018 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760293+0000", 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8666999999999998e-05 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760385+0000", 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1409000000000005e-05 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10545 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 43008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727048+0000", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6359719770000001, 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033462442000000002, 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10545 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727048+0000", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727048+0000", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727047+0000", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727055+0000", 2024-09-21T14:21:33.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2919999999999998e-06 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727061+0000", 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727065+0000", 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7600000000000002e-06 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760413+0000", 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033347732999999997 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760460+0000", 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6918e-05 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760510+0000", 2024-09-21T14:21:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0340000000000003e-05 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10550 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727070+0000", 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.635950062, 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033576481999999998, 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.381 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10550 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727070+0000", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727070+0000", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727069+0000", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727072+0000", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2879999999999999e-06 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727077+0000", 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6500000000000001e-06 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727082+0000", 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5460000000000002e-06 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760537+0000", 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033455458 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760594+0000", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6268999999999999e-05 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760646+0000", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2735000000000002e-05 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10552 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 112640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727086+0000", 2024-09-21T14:21:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6359342830000001, 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033678014999999999, 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10552 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727086+0000", 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727086+0000", 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727085+0000", 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727087+0000", 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0339999999999999e-06 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727094+0000", 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4999999999999996e-06 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727098+0000", 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2119999999999997e-06 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760671+0000", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033572600000000001 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760714+0000", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3231e-05 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760764+0000", 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.389 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10555 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 133120~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727103+0000", 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6359172989999999, 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033797470000000003, 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.390 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10555 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727103+0000", 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727103+0000", 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727102+0000", 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727104+0000", 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0549999999999998e-06 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727111+0000", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7730000000000002e-06 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727116+0000", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.437e-06 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760788+0000", 2024-09-21T14:21:33.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033672778 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760847+0000", 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8801999999999997e-05 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760900+0000", 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2972e-05 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.394 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10557 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 157696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727120+0000", 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6359003030000001, 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033900766999999998, 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10557 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.395 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727120+0000", 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727120+0000", 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727119+0000", 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727121+0000", 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0760000000000001e-06 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727127+0000", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.31e-06 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727131+0000", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3989999999999997e-06 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760927+0000", 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033795332999999997 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.760971+0000", 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4338999999999997e-05 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761020+0000", 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9629999999999997e-05 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.399 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10562 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 202752~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727135+0000", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358847889999999, 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034031161999999997, 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10562 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727135+0000", 2024-09-21T14:21:33.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727135+0000", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727135+0000", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727137+0000", 2024-09-21T14:21:33.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.978e-06 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727143+0000", 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727148+0000", 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0470000000000001e-06 2024-09-21T14:21:33.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761049+0000", 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033900815000000001 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761112+0000", 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2799000000000006e-05 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761166+0000", 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4712000000000001e-05 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10564 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727152+0000", 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358681819999998, 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034135301999999999, 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.404 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10564 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727152+0000", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727152+0000", 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727151+0000", 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727154+0000", 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.069e-06 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727159+0000", 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8560000000000003e-06 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727163+0000", 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0770000000000001e-06 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761187+0000", 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034023618999999998 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761235+0000", 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7409e-05 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761287+0000", 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2621999999999999e-05 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10568 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 247808~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.408 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727167+0000", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358528240000001, 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034265491000000002, 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10568 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727167+0000", 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727167+0000", 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727167+0000", 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727168+0000", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.491e-06 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727171+0000", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727174+0000", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.526e-06 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761319+0000", 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034145174 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761373+0000", 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4178999999999997e-05 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761433+0000", 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9298e-05 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10570 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 272384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727176+0000", 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358438899999999, 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034380013000000001, 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10570 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.413 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727176+0000", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727176+0000", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727176+0000", 2024-09-21T14:21:33.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727177+0000", 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.207e-06 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727180+0000", 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9849999999999998e-06 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727183+0000", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5399999999999998e-06 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761456+0000", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034273519000000002 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761505+0000", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9126999999999999e-05 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761556+0000", 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0856000000000002e-05 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10575 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 317440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727185+0000", 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.635835379, 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034492733999999997, 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10575 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727185+0000", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727185+0000", 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727184+0000", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727186+0000", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3379999999999999e-06 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727189+0000", 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9909999999999998e-06 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727191+0000", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.43e-06 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761579+0000", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034387676999999998 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761626+0000", 2024-09-21T14:21:33.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6891000000000002e-05 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761677+0000", 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1631000000000001e-05 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10576 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 342016~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727194+0000", 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358261380000001, 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034607727999999997, 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10576 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727194+0000", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727194+0000", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727194+0000", 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727195+0000", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.218e-06 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727198+0000", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8650000000000001e-06 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727200+0000", 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2970000000000002e-06 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761700+0000", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034500279000000002 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761751+0000", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0630000000000001e-05 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761802+0000", 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0658999999999997e-05 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10581 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 362496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727202+0000", 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358178140000001, 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034718609999999997, 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10581 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727202+0000", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727202+0000", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727202+0000", 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727203+0000", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.209e-06 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727206+0000", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9330000000000001e-06 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727208+0000", 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:21:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761827+0000", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034618942999999999 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761871+0000", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3884999999999997e-05 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761921+0000", 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9558999999999998e-05 2024-09-21T14:21:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10582 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 387072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727210+0000", 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6358095939999999, 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034834783000000001, 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10582 2024-09-21T14:21:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727210+0000", 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727210+0000", 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727210+0000", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727215+0000", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7280000000000003e-06 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727224+0000", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4080000000000004e-06 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727227+0000", 2024-09-21T14:21:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0920000000000002e-06 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761942+0000", 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034715039000000003 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.761993+0000", 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1017000000000003e-05 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762045+0000", 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1722999999999999e-05 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10586 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 432128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727230+0000", 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6357901930000001, 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034933815999999999, 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10586 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727230+0000", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727230+0000", 2024-09-21T14:21:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727230+0000", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727231+0000", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3650000000000001e-06 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727235+0000", 2024-09-21T14:21:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.014e-06 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727237+0000", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5239999999999999e-06 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762069+0000", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034831235000000002 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762113+0000", 2024-09-21T14:21:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4739e-05 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762164+0000", 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.02e-05 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10589 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 456704~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727239+0000", 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6357805440000002, 2024-09-21T14:21:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035050932999999999, 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10589 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727239+0000", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727239+0000", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727239+0000", 2024-09-21T14:21:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727241+0000", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.364e-06 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727247+0000", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4649999999999999e-06 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727250+0000", 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.65e-06 2024-09-21T14:21:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762186+0000", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034936230999999998 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762238+0000", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1603e-05 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762290+0000", 2024-09-21T14:21:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2825999999999999e-05 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10592 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 477184~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727252+0000", 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.635767972, 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035167412000000002, 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10592 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727252+0000", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727252+0000", 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727252+0000", 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727253+0000", 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.282e-06 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727257+0000", 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.579e-06 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727259+0000", 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.441e-06 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762320+0000", 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035060713 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762361+0000", 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1362999999999997e-05 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762419+0000", 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8276999999999998e-05 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10594 3.6 3:632f6a7c:::rbd_data.1121cf46bdbd.0000000000000012:7 [read 501760~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.727261+0000", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.6357589150000003, 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035262069, 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10594 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727261+0000", 2024-09-21T14:21:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727261+0000", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727261+0000", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727262+0000", 2024-09-21T14:21:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1030000000000001e-06 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727265+0000", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8439999999999998e-06 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.727267+0000", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4150000000000002e-06 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762437+0000", 2024-09-21T14:21:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035169262 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762484+0000", 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7203000000000002e-05 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.762523+0000", 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9443000000000002e-05 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.452 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.453 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:33.453 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:33.453+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:33.554 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:33.554+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:33.555 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.555 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:33.555 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:33.555 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.564 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:33.654 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:33.654+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:33.706 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:33.707 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:33.756 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:33.756+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1075572039999999, 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.818 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:33.819 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:21:33.820 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1075458940000003, 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:33.823 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1075305179999999, 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.827 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.828 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:21:33.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.831 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.10751299, 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:21:33.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074940389999997, 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.836 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.837 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:33.840 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074776589999997, 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:21:33.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:21:33.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.107468431, 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.845 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:21:33.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.849 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074570350000004, 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.850 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:21:33.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074343080000002, 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.854 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074180519999999, 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.858 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1074032970000003, 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:21:33.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:33.866 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1073942429999999, 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.107383199, 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1073677980000003, 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.875 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.876 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1073510999999998, 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:21:33.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.884 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1073354929999999, 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.885 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:33.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:21:33.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1073183709999999, 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.889 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:21:33.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:21:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1072997600000001, 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:21:33.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.1072804300000003, 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.903 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.107260793, 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:33.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:33.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.907 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:33.908 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:33.908 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:33.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:33.909 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:33.909+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:34.010 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:34.010+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:34.011 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.011 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:34.011 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:34.011 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.020 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:34.110 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:34.110+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:34.163 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:34.163 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.172 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:34.215 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:34.215+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:34.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8100 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 442368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8447253889999997, 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013866378, 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.277 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8100 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431780+0000", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431785+0000", 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431793+0000", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4989999999999994e-06 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431799+0000", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9540000000000001e-06 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445497+0000", 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013697611 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445577+0000", 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0795000000000001e-05 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445647+0000", 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9942000000000002e-05 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.281 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.844702249, 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.282 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.285 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8446856069999997, 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.290 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8446605570000001, 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.291 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.294 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:34.295 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8446424050000001, 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:21:34.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.299 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8446191680000004, 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.303 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8445981009999999, 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:21:34.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.308 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8445817409999998, 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.312 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.844551461, 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.313 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8445363229999998, 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.318 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:34.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:21:34.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8445167270000002, 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.322 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:21:34.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.325 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.326 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8445079499999997, 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8444955890000001, 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.331 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:21:34.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:21:34.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8444861719999999, 2024-09-21T14:21:34.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:21:34.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.339 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8444649809999998, 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.340 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:21:34.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.344 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.844447873, 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:21:34.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8444309299999997, 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.349 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:21:34.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:21:34.353 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8444145509999998, 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.354 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:21:34.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:21:34.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.358 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8443731909999999, 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:34.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8443535100000004, 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:21:34.363 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:21:34.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:21:34.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.368 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.369 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone7 from 0xcdd5e20 (overlap 0xcdd5e20) to 0x3dc02a3 2024-09-21T14:21:34.370 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:34.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:34.369+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:34.470 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:34.470+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:34.512 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.512 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:34.512 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:34.513 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.521 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:34.605 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:34.605+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:34.672 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.673 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:34.672+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:34.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:34.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:34.773+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:34.827 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2576 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18350080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.588463127000001, 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018251078, 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2576 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238397+0000", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238400+0000", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.74e-06 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238406+0000", 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2369999999999997e-06 2024-09-21T14:21:34.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238411+0000", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.639e-06 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256452+0000", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018040901000000002 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256536+0000", 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4635999999999994e-05 2024-09-21T14:21:34.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256649+0000", 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00011244299999999999 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.588445780000001, 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.832 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:34.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2304644160000002, 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.837 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:21:34.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:21:34.840 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2304470900000002, 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:21:34.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:21:34.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2304295759999997, 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2304123220000003, 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2303844719999999, 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:21:34.854 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:21:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2303628680000003, 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.859 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:21:34.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.862 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2303458699999998, 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.863 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:34.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:21:34.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.866 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.867 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2303288500000003, 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:34.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:21:34.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.870 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2303113999999997, 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.872 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:21:34.875 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2302942620000001, 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:21:34.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:21:34.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.879 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2302710350000003, 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.880 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:21:34.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2302547610000003, 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.884 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2302303889999999, 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.892 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2302078219999997, 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.893 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:21:34.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:21:34.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2301909010000003, 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:21:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:21:34.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:21:34.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2301744020000003, 2024-09-21T14:21:34.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:21:34.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2301534050000003, 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.907 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:34.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.911 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.2301313470000004, 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:34.912 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:34.913 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.914 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.915 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.916 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.916 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:34.916 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:34.916 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:34.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:1090 trunc from 0xcdd5e20 to 0xc4c8294 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:1094 punch from 0x20831b9 to 0x208bc42, (0x8a89 bytes) 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:1096 write 0xbb52c84 thru 0xbb60f5f (0xe2dc bytes) 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:1097 read 0xa48aa46 thru 0xa490fcc (0x6587 bytes) 2024-09-21T14:21:34.918 INFO:teuthology.orchestra.run.smithi073.stdout:1098 trunc from 0xc4c8294 to 0xcc009e0 2024-09-21T14:21:34.918 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:34.917+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:34.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:34.920 INFO:teuthology.orchestra.run.smithi073.stdout:1099 punch from 0x43e5cc8 to 0x43edf22, (0x825a bytes) 2024-09-21T14:21:34.920 INFO:teuthology.orchestra.run.smithi073.stdout:1100 clone 9 order 22 su 262144 sc 5 2024-09-21T14:21:35.018 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:35.018+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:35.037 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:35.037 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:35.054 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:35.066 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.066 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.070 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:35.070 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:35.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:35.086 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:35.087 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:35.103 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:35.118 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:35.118+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:35.119 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:35.119 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:35.136 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:35.136 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:35.152 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:35.152 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:35.180 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.180 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:21:35.230 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:35.230+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:35.293 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone8 from 0xcc009e0 (overlap 0xc4c8294) to 0x89bf501 2024-09-21T14:21:35.320 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:35.320+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:35.363 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.364 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:35.420 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:35.421+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:35.492 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T14:21:35.523 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:35.522+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:35.535 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.535 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:35.612 INFO:teuthology.orchestra.run.smithi073.stdout:1101 trunc from 0xcc009e0 to 0x592d596 2024-09-21T14:21:35.622 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:35.622+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:35.635 INFO:teuthology.orchestra.run.smithi073.stdout:1102 read 0x5154d26 thru 0x515b2b0 (0x658b bytes) 2024-09-21T14:21:35.649 INFO:teuthology.orchestra.run.smithi073.stdout:1103 write 0xc6177e7 thru 0xc6196bb (0x1ed5 bytes) 2024-09-21T14:21:35.654 INFO:teuthology.orchestra.run.smithi073.stdout:1104 write 0x546b368 thru 0x546fd0b (0x49a4 bytes) 2024-09-21T14:21:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:1105 writesame 0x65a8dd2 thru 0x65b5b0f data_size 0x3f(0xcd3e bytes) 2024-09-21T14:21:35.659 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.665 INFO:teuthology.orchestra.run.smithi073.stdout:1107 writesame 0xcbf816f thru 0xcbf9b25 data_size 0x1(0x19b7 bytes) 2024-09-21T14:21:35.673 INFO:teuthology.orchestra.run.smithi073.stdout:1109 write 0xe4ad691 thru 0xe4b0bd6 (0x3546 bytes) 2024-09-21T14:21:35.677 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.677 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.677 INFO:teuthology.orchestra.run.smithi073.stdout:1112 write 0x5e03dd3 thru 0x5e0b0e3 (0x7311 bytes) 2024-09-21T14:21:35.679 INFO:teuthology.orchestra.run.smithi073.stdout:1113 read 0x98fd1ab thru 0x9900efe (0x3d54 bytes) 2024-09-21T14:21:35.679 INFO:teuthology.orchestra.run.smithi073.stdout:1114 punch from 0xbcb37f4 to 0xbcc2121, (0xe92d bytes) 2024-09-21T14:21:35.679 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.679 INFO:teuthology.orchestra.run.smithi073.stdout:1116 read 0xbb0c616 thru 0xbb18e61 (0xc84c bytes) 2024-09-21T14:21:35.679 INFO:teuthology.orchestra.run.smithi073.stdout:1117 write 0x2760869 thru 0x2764ae4 (0x427c bytes) 2024-09-21T14:21:35.686 INFO:teuthology.orchestra.run.smithi073.stdout:1118 punch from 0x45595a6 to 0x456182e, (0x8288 bytes) 2024-09-21T14:21:35.686 INFO:teuthology.orchestra.run.smithi073.stdout:1119 read 0xc0556fa thru 0xc05732c (0x1c33 bytes) 2024-09-21T14:21:35.686 INFO:teuthology.orchestra.run.smithi073.stdout:1120 writesame 0x2c9f5a1 thru 0x2cace54 data_size 0x45(0xd8b4 bytes) 2024-09-21T14:21:35.701 INFO:teuthology.orchestra.run.smithi073.stdout:1121 read 0xbc05bae thru 0xbc06c78 (0x10cb bytes) 2024-09-21T14:21:35.702 INFO:teuthology.orchestra.run.smithi073.stdout:1124 read 0xae2b70e thru 0xae381b5 (0xcaa8 bytes) 2024-09-21T14:21:35.705 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.705 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.716 INFO:teuthology.orchestra.run.smithi073.stdout:1125 trunc from 0xe4b0bd7 to 0x20bdda1 2024-09-21T14:21:35.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:1126 punch from 0xbaac50 to 0xbac1ed, (0x159d bytes) 2024-09-21T14:21:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:1128 read 0x19f9dbe thru 0x1a062bd (0xc500 bytes) 2024-09-21T14:21:35.748 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:35.747+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:35.748 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.759 INFO:teuthology.orchestra.run.smithi073.stdout:1130 write 0x6a20227 thru 0x6a28c93 (0x8a6d bytes) 2024-09-21T14:21:35.762 INFO:teuthology.orchestra.run.smithi073.stdout:1131 read 0x1c3d343 thru 0x1c3ffea (0x2ca8 bytes) 2024-09-21T14:21:35.763 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.763 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.763 INFO:teuthology.orchestra.run.smithi073.stdout:1134 punch from 0xc3877a to 0xc41ed0, (0x9756 bytes) 2024-09-21T14:21:35.763 INFO:teuthology.orchestra.run.smithi073.stdout:1135 write 0x4cd3967 thru 0x4cdfcc1 (0xc35b bytes) 2024-09-21T14:21:35.766 INFO:teuthology.orchestra.run.smithi073.stdout:1136 read 0x6693ffa thru 0x66a266e (0xe675 bytes) 2024-09-21T14:21:35.782 INFO:teuthology.orchestra.run.smithi073.stdout:1137 write 0xed598ee thru 0xed65a3e (0xc151 bytes) 2024-09-21T14:21:35.786 INFO:teuthology.orchestra.run.smithi073.stdout:1138 trunc from 0xed65a3f to 0xe24cf99 2024-09-21T14:21:35.792 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.792 INFO:teuthology.orchestra.run.smithi073.stdout:1140 punch from 0x4ea790b to 0x4eb0428, (0x8b1d bytes) 2024-09-21T14:21:35.792 INFO:teuthology.orchestra.run.smithi073.stdout:1141 write 0xc8ad5f3 thru 0xc8b1eba (0x48c8 bytes) 2024-09-21T14:21:35.796 INFO:teuthology.orchestra.run.smithi073.stdout:1142 punch from 0x1b7145b to 0x1b7e194, (0xcd39 bytes) 2024-09-21T14:21:35.796 INFO:teuthology.orchestra.run.smithi073.stdout:1143 punch from 0x1f2b2c4 to 0x1f344b1, (0x91ed bytes) 2024-09-21T14:21:35.802 INFO:teuthology.orchestra.run.smithi073.stdout:1144 write 0xeb44522 thru 0xeb465f7 (0x20d6 bytes) 2024-09-21T14:21:35.805 INFO:teuthology.orchestra.run.smithi073.stdout:1145 write 0xd8ceee2 thru 0xd8d2bd8 (0x3cf7 bytes) 2024-09-21T14:21:35.806 INFO:teuthology.orchestra.run.smithi073.stdout:1146 writesame 0xcc75a2f thru 0xcc7d487 data_size 0x1(0x7a59 bytes) 2024-09-21T14:21:35.821 INFO:teuthology.orchestra.run.smithi073.stdout:1147 write 0x24f9cd7 thru 0x2504031 (0xa35b bytes) 2024-09-21T14:21:35.823 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:35.823 INFO:teuthology.orchestra.run.smithi073.stdout:1151 writesame 0xb52c8e2 thru 0xb535551 data_size 0x70(0x8c70 bytes) 2024-09-21T14:21:35.823 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:35.823+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:35.826 INFO:teuthology.orchestra.run.smithi073.stdout:1152 writesame 0x213d4d7 thru 0x213fa94 data_size 0x1(0x25be bytes) 2024-09-21T14:21:35.831 INFO:teuthology.orchestra.run.smithi073.stdout:1153 clone 10 order 21 su 16384 sc 9 2024-09-21T14:21:35.859 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:35.859 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:35.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:35.925 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:35.925+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:36.010 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.011 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:36.063 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:36.063+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:36.121 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.121 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.125 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:36.126+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:36.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:36.226 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:36.227+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:36.277 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.277 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:36.301 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone9 from 0xeb465f8 (overlap 0x20bdda1) to 0x1a317b4 2024-09-21T14:21:36.327 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:36.327+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:36.395 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.396 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.402 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T14:21:36.405 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:36.428 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:36.428+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:36.529 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:36.529+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:36.579 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.579 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:36.630 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:36.630+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:36.653 INFO:teuthology.orchestra.run.smithi073.stdout:1154 read 0xbc3472a thru 0xbc37df5 (0x36cc bytes) 2024-09-21T14:21:36.654 INFO:teuthology.orchestra.run.smithi073.stdout:1155 read 0xd995495 thru 0xd99edbc (0x9928 bytes) 2024-09-21T14:21:36.670 INFO:teuthology.orchestra.run.smithi073.stdout:1156 trunc from 0xeb465f8 to 0x1dc3126 2024-09-21T14:21:36.711 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.711 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:36.737 INFO:teuthology.orchestra.run.smithi073.stdout:1157 read 0x1b85d0c thru 0x1b92843 (0xcb38 bytes) 2024-09-21T14:21:36.738 INFO:teuthology.orchestra.run.smithi073.stdout:1158 punch from 0x1823180 to 0x18303b1, (0xd231 bytes) 2024-09-21T14:21:36.738 INFO:teuthology.orchestra.run.smithi073.stdout:1160 read 0x4165a8 thru 0x4223e1 (0xbe3a bytes) 2024-09-21T14:21:36.754 INFO:teuthology.orchestra.run.smithi073.stdout:1161 write 0xab3e98c thru 0xab42a98 (0x410d bytes) 2024-09-21T14:21:36.758 INFO:teuthology.orchestra.run.smithi073.stdout:1162 write 0x8cde5e3 thru 0x8cdfe1c (0x183a bytes) 2024-09-21T14:21:36.763 INFO:teuthology.orchestra.run.smithi073.stdout:1164 read 0x3a637b8 thru 0x3a6a76f (0x6fb8 bytes) 2024-09-21T14:21:36.763 INFO:teuthology.orchestra.run.smithi073.stdout:1165 punch from 0x9974f07 to 0x997d616, (0x870f bytes) 2024-09-21T14:21:36.763 INFO:teuthology.orchestra.run.smithi073.stdout:1166 read 0x6329156 thru 0x632e7b1 (0x565c bytes) 2024-09-21T14:21:36.766 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:36.767+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:36.769 INFO:teuthology.orchestra.run.smithi073.stdout:1170 write 0xb33db93 thru 0xb34cef0 (0xf35e bytes) 2024-09-21T14:21:36.773 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:36.777 INFO:teuthology.orchestra.run.smithi073.stdout:1172 trunc from 0xb34cef1 to 0xeb10f47 2024-09-21T14:21:36.784 INFO:teuthology.orchestra.run.smithi073.stdout:1173 write 0x20a99af thru 0x20abc33 (0x2285 bytes) 2024-09-21T14:21:36.786 INFO:teuthology.orchestra.run.smithi073.stdout:1174 punch from 0x42c6796 to 0x42cb8c1, (0x512b bytes) 2024-09-21T14:21:36.786 INFO:teuthology.orchestra.run.smithi073.stdout:1176 read 0x5271225 thru 0x527640d (0x51e9 bytes) 2024-09-21T14:21:36.786 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:36.786 INFO:teuthology.orchestra.run.smithi073.stdout:1179 write 0xc047dde thru 0xc04d3a0 (0x55c3 bytes) 2024-09-21T14:21:36.801 INFO:teuthology.orchestra.run.smithi073.stdout:1182 trunc from 0xeb10f47 to 0x7ba02be 2024-09-21T14:21:36.811 INFO:teuthology.orchestra.run.smithi073.stdout:1184 write 0x7016a3a thru 0x7022c54 (0xc21b bytes) 2024-09-21T14:21:36.820 INFO:teuthology.orchestra.run.smithi073.stdout:1185 read 0x5fdcfba thru 0x5fe04f5 (0x353c bytes) 2024-09-21T14:21:36.824 INFO:teuthology.orchestra.run.smithi073.stdout:1186 write 0x7c6182a thru 0x7c69ca8 (0x847f bytes) 2024-09-21T14:21:36.828 INFO:teuthology.orchestra.run.smithi073.stdout:1187 punch from 0x4ee0eb6 to 0x4ee6804, (0x594e bytes) 2024-09-21T14:21:36.829 INFO:teuthology.orchestra.run.smithi073.stdout:1188 writesame 0x57718d4 thru 0x577628a data_size 0x1(0x49b7 bytes) 2024-09-21T14:21:36.832 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:36.832+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:36.832 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:36.832 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:36.835 INFO:teuthology.orchestra.run.smithi073.stdout:1189 writesame 0x177fa35 thru 0x17866c1 data_size 0x3(0x6c8d bytes) 2024-09-21T14:21:36.839 INFO:teuthology.orchestra.run.smithi073.stdout:1190 punch from 0x16bb543 to 0x16bfe1e, (0x48db bytes) 2024-09-21T14:21:36.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:36.845 INFO:teuthology.orchestra.run.smithi073.stdout:1191 trunc from 0x7c69ca9 to 0xc6c5c30 2024-09-21T14:21:36.858 INFO:teuthology.orchestra.run.smithi073.stdout:1192 trunc from 0xc6c5c30 to 0x5bb3f8b 2024-09-21T14:21:36.880 INFO:teuthology.orchestra.run.smithi073.stdout:1193 write 0xb073957 thru 0xb0787c6 (0x4e70 bytes) 2024-09-21T14:21:36.884 INFO:teuthology.orchestra.run.smithi073.stdout:1194 flatten 2024-09-21T14:21:36.933 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:36.933+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:36.964 INFO:teuthology.orchestra.run.smithi073.stdout:1195 write 0xe47514d thru 0xe47889e (0x3752 bytes) 2024-09-21T14:21:36.971 INFO:teuthology.orchestra.run.smithi073.stdout:1196 write 0x70e8176 thru 0x70e94de (0x1369 bytes) 2024-09-21T14:21:36.973 INFO:teuthology.orchestra.run.smithi073.stdout:1198 write 0x9757ae6 thru 0x9767672 (0xfb8d bytes) 2024-09-21T14:21:36.983 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:36.983 INFO:teuthology.orchestra.run.smithi073.stdout:1202 writesame 0x824cb0e thru 0x824cf59 data_size 0x1(0x44c bytes) 2024-09-21T14:21:36.988 INFO:teuthology.orchestra.run.smithi073.stdout:1203 read 0x73ba799 thru 0x73bf261 (0x4ac9 bytes) 2024-09-21T14:21:36.988 INFO:teuthology.orchestra.run.smithi073.stdout:1204 writesame 0xe14bbe thru 0xe1bba2 data_size 0x11(0x6fe5 bytes) 2024-09-21T14:21:36.990 INFO:teuthology.orchestra.run.smithi073.stdout:1205 write 0x1f3a0e4 thru 0x1f4161a (0x7537 bytes) 2024-09-21T14:21:36.991 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:36.991 INFO:teuthology.orchestra.run.smithi073.stdout:1209 write 0x45462ab thru 0x4548426 (0x217c bytes) 2024-09-21T14:21:36.994 INFO:teuthology.orchestra.run.smithi073.stdout:1213 write 0x61162e1 thru 0x611f06c (0x8d8c bytes) 2024-09-21T14:21:37.000 INFO:teuthology.orchestra.run.smithi073.stdout:1214 read 0x342c6fc thru 0x342da28 (0x132d bytes) 2024-09-21T14:21:37.007 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:37.007 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:37.012 INFO:teuthology.orchestra.run.smithi073.stdout:1215 trunc from 0xe47889f to 0x3d95b63 2024-09-21T14:21:37.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:37.028 INFO:teuthology.orchestra.run.smithi073.stdout:1217 punch from 0x1010117 to 0x1012f9f, (0x2e88 bytes) 2024-09-21T14:21:37.033 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:37.033+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:37.035 INFO:teuthology.orchestra.run.smithi073.stdout:1218 write 0x43f0e91 thru 0x43f2079 (0x11e9 bytes) 2024-09-21T14:21:37.053 INFO:teuthology.orchestra.run.smithi073.stdout:1219 write 0xbe8b1e6 thru 0xbe92292 (0x70ad bytes) 2024-09-21T14:21:37.067 INFO:teuthology.orchestra.run.smithi073.stdout:1223 trunc from 0xbe92293 to 0x59b1817 2024-09-21T14:21:37.076 INFO:teuthology.orchestra.run.smithi073.stdout:1225 writesame 0x4eda801 thru 0x4eea727 data_size 0x1(0xff27 bytes) 2024-09-21T14:21:37.094 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.094 INFO:teuthology.orchestra.run.smithi073.stdout:1229 punch from 0x10843b4 to 0x108c7b3, (0x83ff bytes) 2024-09-21T14:21:37.095 INFO:teuthology.orchestra.run.smithi073.stdout:1230 read 0x58617fc thru 0x586b64d (0x9e52 bytes) 2024-09-21T14:21:37.095 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.095 INFO:teuthology.orchestra.run.smithi073.stdout:1234 read 0x4f3a71 thru 0x4f98eb (0x5e7b bytes) 2024-09-21T14:21:37.096 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.096 INFO:teuthology.orchestra.run.smithi073.stdout:1237 trunc from 0x59b1817 to 0x64d836a 2024-09-21T14:21:37.106 INFO:teuthology.orchestra.run.smithi073.stdout:1239 writesame 0x72709c8 thru 0x727ac93 data_size 0x2e(0xa2cc bytes) 2024-09-21T14:21:37.110 INFO:teuthology.orchestra.run.smithi073.stdout:1240 write 0x79809cb thru 0x79853c5 (0x49fb bytes) 2024-09-21T14:21:37.114 INFO:teuthology.orchestra.run.smithi073.stdout:1241 punch from 0x3a46878 to 0x3a48b68, (0x22f0 bytes) 2024-09-21T14:21:37.119 INFO:teuthology.orchestra.run.smithi073.stdout:1242 trunc from 0x79853c6 to 0x3708d2d 2024-09-21T14:21:37.135 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:37.135+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:37.149 INFO:teuthology.orchestra.run.smithi073.stdout:1243 write 0xe041b4a thru 0xe04c3c4 (0xa87b bytes) 2024-09-21T14:21:37.154 INFO:teuthology.orchestra.run.smithi073.stdout:1244 read 0xbb201c thru 0xbbbda2 (0x9d87 bytes) 2024-09-21T14:21:37.155 INFO:teuthology.orchestra.run.smithi073.stdout:1245 read 0xab573b9 thru 0xab586ef (0x1337 bytes) 2024-09-21T14:21:37.155 INFO:teuthology.orchestra.run.smithi073.stdout:1246 writesame 0xe668a2 thru 0xe762d5 data_size 0x86(0xfa34 bytes) 2024-09-21T14:21:37.156 INFO:teuthology.orchestra.run.smithi073.stdout:1247 write 0x38d3737 thru 0x38d9c55 (0x651f bytes) 2024-09-21T14:21:37.162 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.162 INFO:teuthology.orchestra.run.smithi073.stdout:1250 punch from 0x7c49963 to 0x7c53db3, (0xa450 bytes) 2024-09-21T14:21:37.162 INFO:teuthology.orchestra.run.smithi073.stdout:1251 read 0xdd423b2 thru 0xdd4eb78 (0xc7c7 bytes) 2024-09-21T14:21:37.162 INFO:teuthology.orchestra.run.smithi073.stdout:1252 punch from 0x659683b to 0x6599b19, (0x32de bytes) 2024-09-21T14:21:37.162 INFO:teuthology.orchestra.run.smithi073.stdout:1254 writesame 0x818276e thru 0x818e686 data_size 0x17(0xbf19 bytes) 2024-09-21T14:21:37.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:37.168 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:37.168 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:37.171 INFO:teuthology.orchestra.run.smithi073.stdout:1255 trunc from 0xe04c3c5 to 0xb978c21 2024-09-21T14:21:37.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:37.180 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xee1e946 2024-09-21T14:21:37.184 INFO:teuthology.orchestra.run.smithi073.stdout:1256 trunc from 0xb978c21 to 0xee1e946 2024-09-21T14:21:37.189 INFO:teuthology.orchestra.run.smithi073.stdout:1257 read 0x88d198d thru 0x88d3bef (0x2263 bytes) 2024-09-21T14:21:37.191 INFO:teuthology.orchestra.run.smithi073.stdout:1258 trunc from 0xee1e946 to 0xd447e66 2024-09-21T14:21:37.202 INFO:teuthology.orchestra.run.smithi073.stdout:1259 writesame 0xe086434 thru 0xe0917c0 data_size 0x1d(0xb38d bytes) 2024-09-21T14:21:37.203 INFO:teuthology.orchestra.run.smithi073.stdout:1260 trunc from 0xe0917c1 to 0xead0e75 2024-09-21T14:21:37.208 INFO:teuthology.orchestra.run.smithi073.stdout:1262 write 0xbe3f086 thru 0xbe4d5da (0xe555 bytes) 2024-09-21T14:21:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:1263 read 0x2e06855 thru 0x2e10e3c (0xa5e8 bytes) 2024-09-21T14:21:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:1266 read 0x6398518 thru 0x639eeca (0x69b3 bytes) 2024-09-21T14:21:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:1268 read 0xdb5242 thru 0xdb7665 (0x2424 bytes) 2024-09-21T14:21:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:1269 read 0x8b9c673 thru 0x8ba8456 (0xbde4 bytes) 2024-09-21T14:21:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:1270 read 0x5964cc0 thru 0x596c928 (0x7c69 bytes) 2024-09-21T14:21:37.218 INFO:teuthology.orchestra.run.smithi073.stdout:1273 trunc from 0xead0e75 to 0xcad10b2 2024-09-21T14:21:37.223 INFO:teuthology.orchestra.run.smithi073.stdout:1274 read 0x693f042 thru 0x694691e (0x78dd bytes) 2024-09-21T14:21:37.223 INFO:teuthology.orchestra.run.smithi073.stdout:1275 write 0xb77ad80 thru 0xb77f61d (0x489e bytes) 2024-09-21T14:21:37.226 INFO:teuthology.orchestra.run.smithi073.stdout:1277 read 0x47ecabc thru 0x47f01bc (0x3701 bytes) 2024-09-21T14:21:37.226 INFO:teuthology.orchestra.run.smithi073.stdout:1278 punch from 0x44299e8 to 0x4433520, (0x9b38 bytes) 2024-09-21T14:21:37.226 INFO:teuthology.orchestra.run.smithi073.stdout:1279 write 0x1684949 thru 0x16898cf (0x4f87 bytes) 2024-09-21T14:21:37.228 INFO:teuthology.orchestra.run.smithi073.stdout:1280 read 0xb2e8b33 thru 0xb2f10d6 (0x85a4 bytes) 2024-09-21T14:21:37.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.228 INFO:teuthology.orchestra.run.smithi073.stdout:1287 write 0x85c3920 thru 0x85ceec4 (0xb5a5 bytes) 2024-09-21T14:21:37.232 INFO:teuthology.orchestra.run.smithi073.stdout:1288 write 0x71d8389 thru 0x71e44cf (0xc147 bytes) 2024-09-21T14:21:37.236 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:37.236+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:37.237 INFO:teuthology.orchestra.run.smithi073.stdout:1289 read 0x29cbf83 thru 0x29d11be (0x523c bytes) 2024-09-21T14:21:37.237 INFO:teuthology.orchestra.run.smithi073.stdout:1292 punch from 0x42a7a91 to 0x42b78f8, (0xfe67 bytes) 2024-09-21T14:21:37.237 INFO:teuthology.orchestra.run.smithi073.stdout:1293 writesame 0x2985d70 thru 0x298d36b data_size 0x1(0x75fc bytes) 2024-09-21T14:21:37.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:21:37.256 INFO:teuthology.orchestra.run.smithi073.stdout:1294 write 0xe9a97c0 thru 0xe9b4ab3 (0xb2f4 bytes) 2024-09-21T14:21:37.262 INFO:teuthology.orchestra.run.smithi073.stdout:1295 write 0x59fe07 thru 0x5ac553 (0xc74d bytes) 2024-09-21T14:21:37.276 INFO:teuthology.orchestra.run.smithi073.stdout:1298 trunc from 0xe9b4ab4 to 0x3fc7779 2024-09-21T14:21:37.291 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.301 INFO:teuthology.orchestra.run.smithi073.stdout:1300 trunc from 0x3fc7779 to 0xd098323 2024-09-21T14:21:37.309 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:37.309 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:37.311 INFO:teuthology.orchestra.run.smithi073.stdout:1301 read 0x69720d7 thru 0x6981eae (0xfdd8 bytes) 2024-09-21T14:21:37.320 INFO:teuthology.orchestra.run.smithi073.stdout:1302 trunc from 0xd098323 to 0x5cfdd8e 2024-09-21T14:21:37.327 INFO:teuthology.orchestra.run.smithi073.stdout:1303 trunc from 0x5cfdd8e to 0x6b9777b 2024-09-21T14:21:37.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:37.327 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:37.328 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:37.328 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:37.337 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:37.337+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:37.337 INFO:teuthology.orchestra.run.smithi073.stdout:1306 trunc from 0x6b9777b to 0x9c15fae 2024-09-21T14:21:37.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:37.341 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.341 INFO:teuthology.orchestra.run.smithi073.stdout:1308 read 0x162e17c thru 0x163b2b1 (0xd136 bytes) 2024-09-21T14:21:37.343 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.343 INFO:teuthology.orchestra.run.smithi073.stdout:1311 punch from 0x4a4eef3 to 0x4a5b60c, (0xc719 bytes) 2024-09-21T14:21:37.343 INFO:teuthology.orchestra.run.smithi073.stdout:1312 read 0x3d5c41f thru 0x3d6590c (0x94ee bytes) 2024-09-21T14:21:37.343 INFO:teuthology.orchestra.run.smithi073.stdout:1313 writesame 0x1e437c8 thru 0x1e4b2ed data_size 0xb(0x7b26 bytes) 2024-09-21T14:21:37.345 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:37.345 INFO:teuthology.orchestra.run.smithi073.stdout:1316 writesame 0x9754afb thru 0x9759a3f data_size 0xd(0x4f45 bytes) 2024-09-21T14:21:37.348 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:37.348 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:37.357 INFO:teuthology.orchestra.run.smithi073.stdout:1317 writesame 0xc480f85 thru 0xc48e63c data_size 0x4(0xd6b8 bytes) 2024-09-21T14:21:37.357 INFO:teuthology.orchestra.run.smithi073.stdout:1318 read 0x97f2cd4 thru 0x97f875f (0x5a8c bytes) 2024-09-21T14:21:37.357 INFO:teuthology.orchestra.run.smithi073.stdout:1319 writesame 0xa0f7015 thru 0xa1045e1 data_size 0x7(0xd5cd bytes) 2024-09-21T14:21:37.368 INFO:teuthology.orchestra.run.smithi073.stdout:1320 write 0x60be90f thru 0x60c4cea (0x63dc bytes) 2024-09-21T14:21:37.371 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:37.371 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:37.375 INFO:teuthology.orchestra.run.smithi073.stdout:1321 write 0xa53e23e thru 0xa54661f (0x83e2 bytes) 2024-09-21T14:21:37.380 INFO:teuthology.orchestra.run.smithi073.stdout:1323 read 0x6e33c63 thru 0x6e37df7 (0x4195 bytes) 2024-09-21T14:21:37.380 INFO:teuthology.orchestra.run.smithi073.stdout:1324 read 0xbf7e406 thru 0xbf8a010 (0xbc0b bytes) 2024-09-21T14:21:37.380 INFO:teuthology.orchestra.run.smithi073.stdout:1325 read 0x5a43bb7 thru 0x5a458cc (0x1d16 bytes) 2024-09-21T14:21:37.380 INFO:teuthology.orchestra.run.smithi073.stdout:1326 clone 11 order 25 su 8388608 sc 14 2024-09-21T14:21:37.399 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:37.430 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:37.430 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:37.437 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:37.437+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:37.443 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:37.443 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:37.444 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:37.444 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:37.444 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:37.447 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:37.447 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:37.452 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:37.463 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:37.463 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:37.538 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:37.538+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8446190739999997, 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:21:37.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.844607764, 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:21:37.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:37.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:21:37.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:21:37.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445923879999997, 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:21:37.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:21:37.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445748599999998, 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.567 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:21:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:21:37.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445559090000003, 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:21:37.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:21:37.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:21:37.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445395290000004, 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:21:37.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:37.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:21:37.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445303009999998, 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:37.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:21:37.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8445189050000002, 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:21:37.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:21:37.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:21:37.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.844496178, 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.591 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444799219999997, 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:21:37.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:21:37.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444651670000001, 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.602 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:37.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:21:37.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:21:37.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444561129999997, 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.608 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444450689999998, 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:21:37.612 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.616 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444296680000001, 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.617 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:37.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:21:37.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:21:37.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8444129700000005, 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:21:37.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:21:37.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8443973629999997, 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:21:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:37.629 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8443802409999996, 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8443616299999999, 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8443423000000001, 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:21:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:21:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:21:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8443226629999998, 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:21:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:37.646 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:37.647 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:37.648 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:37.647+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:37.748 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:37.749+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:37.790 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:37.790 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:37.791 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:37.791 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:37.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:37.849 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:37.849+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:37.900 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:37.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:37.900 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:37.900 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:37.901 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:37.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:37.950 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:37.950+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:38.051 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:38.051+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8100 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 442368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6205682560000003, 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013866378, 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8100 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431781+0000", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431780+0000", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431785+0000", 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431793+0000", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4989999999999994e-06 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431799+0000", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9540000000000001e-06 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445497+0000", 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013697611 2024-09-21T14:21:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445577+0000", 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0795000000000001e-05 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445647+0000", 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9942000000000002e-05 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6205451160000006, 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6205284740000003, 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6205034240000007, 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:21:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:21:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6204852719999998, 2024-09-21T14:21:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6204620349999992, 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:21:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:21:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6204409680000005, 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:21:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:21:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6204246080000004, 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:21:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:21:38.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203943279999997, 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.087 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:38.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:21:38.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.091 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203791899999995, 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:38.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:21:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.620359594, 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:21:38.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:21:38.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:38.100 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203508170000003, 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:21:38.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:21:38.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.104 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203384560000007, 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:21:38.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203290389999996, 2024-09-21T14:21:38.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6203078479999995, 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:21:38.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:21:38.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6202907399999997, 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:21:38.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6202737969999994, 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:21:38.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:21:38.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6202574179999996, 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6202160580000005, 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:38.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:21:38.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:21:38.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6201963769999992, 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:21:38.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:38.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.138 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:38.139 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:38.153 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:38.153+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:38.227 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:38.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:21:38.244 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:21:38.254 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:38.254+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:38.286 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:38.286 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:38.286 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:38.286 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:38.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:38.379 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:38.379+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:38.440 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:38.440 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:38.440 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:38.440 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:38.440 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:38.449 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:38.455 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T14:21:38.455 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:38.455+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:38.535 INFO:teuthology.orchestra.run.smithi073.stdout:1327 writesame 0xec8b12c thru 0xec8f6db data_size 0x28(0x45b0 bytes) 2024-09-21T14:21:38.556 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:38.556+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:38.558 INFO:teuthology.orchestra.run.smithi073.stdout:1329 read 0xb9b112f thru 0xb9b8ba9 (0x7a7b bytes) 2024-09-21T14:21:38.558 INFO:teuthology.orchestra.run.smithi073.stdout:1331 read 0x126c9bb thru 0x12747b4 (0x7dfa bytes) 2024-09-21T14:21:38.559 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.559 INFO:teuthology.orchestra.run.smithi073.stdout:1333 write 0x35f580e thru 0x36045fa (0xeded bytes) 2024-09-21T14:21:38.564 INFO:teuthology.orchestra.run.smithi073.stdout:1335 read 0x56a75b6 thru 0x56abd64 (0x47af bytes) 2024-09-21T14:21:38.564 INFO:teuthology.orchestra.run.smithi073.stdout:1336 writesame 0xa98f236 thru 0xa99966c data_size 0x51(0xa437 bytes) 2024-09-21T14:21:38.568 INFO:teuthology.orchestra.run.smithi073.stdout:1337 read 0xa77bb47 thru 0xa77f1c9 (0x3683 bytes) 2024-09-21T14:21:38.569 INFO:teuthology.orchestra.run.smithi073.stdout:1339 read 0xc5d6e21 thru 0xc5e6abb (0xfc9b bytes) 2024-09-21T14:21:38.569 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.569 INFO:teuthology.orchestra.run.smithi073.stdout:1343 read 0x74fd4d5 thru 0x750cc2f (0xf75b bytes) 2024-09-21T14:21:38.569 INFO:teuthology.orchestra.run.smithi073.stdout:1345 writesame 0x9fad4a1 thru 0x9fb009b data_size 0x1(0x2bfb bytes) 2024-09-21T14:21:38.576 INFO:teuthology.orchestra.run.smithi073.stdout:1346 punch from 0x5f4bdfe to 0x5f50bbf, (0x4dc1 bytes) 2024-09-21T14:21:38.587 INFO:teuthology.orchestra.run.smithi073.stdout:1347 trunc from 0xec8f6dc to 0x58043ef 2024-09-21T14:21:38.608 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2576 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18350080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 14.369268579, 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018251078, 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2576 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:38.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238397+0000", 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238400+0000", 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.74e-06 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238406+0000", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2369999999999997e-06 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238411+0000", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.639e-06 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256452+0000", 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018040901000000002 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256536+0000", 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4635999999999994e-05 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256649+0000", 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00011244299999999999 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.613 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 14.369251232, 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.614 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0112698679999994, 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:21:38.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.622 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0112525419999994, 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:21:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:21:38.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0112350279999998, 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:21:38.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0112177740000003, 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:21:38.632 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:21:38.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.636 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.011189924, 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.637 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0111683199999995, 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.642 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:38.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:21:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0111513219999999, 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.648 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0111343020000003, 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.653 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:21:38.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0111168520000007, 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.658 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:38.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0110997140000002, 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.662 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:21:38.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:21:38.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:38.667 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0110764870000004, 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:21:38.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0110602130000004, 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:21:38.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.011035841, 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:21:38.676 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:21:38.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:21:38.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0110132739999997, 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.685 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0109963529999995, 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.686 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0109798540000003, 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.690 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:38.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:21:38.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.694 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0109588570000003, 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:21:38.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:38.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:21:38.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:21:38.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.0109367989999996, 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:38.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:38.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:38.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:38.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:38.705 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:38.707 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:38.706+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:38.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:1348 read 0xc96253 thru 0xca0a4f (0xa7fd bytes) 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:1349 write 0xd987083 thru 0xd98beae (0x4e2c bytes) 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:1351 read 0x9b11263 thru 0x9b14570 (0x330e bytes) 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:1352 write 0xb9b47b4 thru 0xb9b564e (0xe9b bytes) 2024-09-21T14:21:38.711 INFO:teuthology.orchestra.run.smithi073.stdout:1353 write 0x7ae693f thru 0x7af1e3d (0xb4ff bytes) 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:1357 write 0x914f487 thru 0x9158784 (0x92fe bytes) 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:1359 write 0xeac669e thru 0xead1f1b (0xb87e bytes) 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:1361 read 0x53310e thru 0x53f0c2 (0xbfb5 bytes) 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:1363 read 0x579997d thru 0x57a28ec (0x8f70 bytes) 2024-09-21T14:21:38.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:1365 write 0xc1ba166 thru 0xc1bc8f9 (0x2794 bytes) 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:1368 read 0xc51c48b thru 0xc52c475 (0xffeb bytes) 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:1369 write 0xacea322 thru 0xaceb70c (0x13eb bytes) 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:1370 punch from 0x34b682f to 0x34bfbdc, (0x93ad bytes) 2024-09-21T14:21:38.713 INFO:teuthology.orchestra.run.smithi073.stdout:1373 punch from 0x3b3941b to 0x3b3ebf9, (0x57de bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1374 read 0x79bd3c1 thru 0x79c92bd (0xbefd bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1375 write 0x9583186 thru 0x9591064 (0xdedf bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1379 read 0xa7c219d thru 0xa7cd261 (0xb0c5 bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1380 write 0xc664886 thru 0xc67471f (0xfe9a bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1381 punch from 0x7fba69f to 0x7fbb4db, (0xe3c bytes) 2024-09-21T14:21:38.714 INFO:teuthology.orchestra.run.smithi073.stdout:1382 writesame 0x94aa47a thru 0x94aa85a data_size 0x1(0x3e1 bytes) 2024-09-21T14:21:38.715 INFO:teuthology.orchestra.run.smithi073.stdout:1383 write 0x9892411 thru 0x989f32c (0xcf1c bytes) 2024-09-21T14:21:38.715 INFO:teuthology.orchestra.run.smithi073.stdout:1385 trunc from 0xead1f1c to 0x2aaa28 2024-09-21T14:21:38.727 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.745 INFO:teuthology.orchestra.run.smithi073.stdout:1387 write 0xa7559f5 thru 0xa758b74 (0x3180 bytes) 2024-09-21T14:21:38.748 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.748 INFO:teuthology.orchestra.run.smithi073.stdout:1390 read 0x4408a99 thru 0x441479f (0xbd07 bytes) 2024-09-21T14:21:38.748 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.748 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.748 INFO:teuthology.orchestra.run.smithi073.stdout:1394 write 0x9f6dad0 thru 0x9f7a833 (0xcd64 bytes) 2024-09-21T14:21:38.752 INFO:teuthology.orchestra.run.smithi073.stdout:1395 write 0x317228b thru 0x3176765 (0x44db bytes) 2024-09-21T14:21:38.753 INFO:teuthology.orchestra.run.smithi073.stdout:1396 write 0x6efc019 thru 0x6efe080 (0x2068 bytes) 2024-09-21T14:21:38.756 INFO:teuthology.orchestra.run.smithi073.stdout:1399 write 0x50b0a4b thru 0x50b5ac7 (0x507d bytes) 2024-09-21T14:21:38.760 INFO:teuthology.orchestra.run.smithi073.stdout:1403 read 0x81b28eb thru 0x81b6d4e (0x4464 bytes) 2024-09-21T14:21:38.760 INFO:teuthology.orchestra.run.smithi073.stdout:1404 read 0x12f320c thru 0x12f6e37 (0x3c2c bytes) 2024-09-21T14:21:38.760 INFO:teuthology.orchestra.run.smithi073.stdout:1406 write 0x6e90695 thru 0x6e985f1 (0x7f5d bytes) 2024-09-21T14:21:38.767 INFO:teuthology.orchestra.run.smithi073.stdout:1407 write 0xae148f9 thru 0xae16ecb (0x25d3 bytes) 2024-09-21T14:21:38.769 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.776 INFO:teuthology.orchestra.run.smithi073.stdout:1409 trunc from 0xae16ecc to 0x499e088 2024-09-21T14:21:38.798 INFO:teuthology.orchestra.run.smithi073.stdout:1410 write 0x4adffe7 thru 0x4aeb23e (0xb258 bytes) 2024-09-21T14:21:38.807 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:38.807+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:38.815 INFO:teuthology.orchestra.run.smithi073.stdout:1411 writesame 0xbeb0d1b thru 0xbeb1339 data_size 0x1(0x61f bytes) 2024-09-21T14:21:38.817 INFO:teuthology.orchestra.run.smithi073.stdout:1412 trunc from 0xbeb133a to 0xa56378f 2024-09-21T14:21:38.825 INFO:teuthology.orchestra.run.smithi073.stdout:1414 read 0x4be628b thru 0x4bf0642 (0xa3b8 bytes) 2024-09-21T14:21:38.825 INFO:teuthology.orchestra.run.smithi073.stdout:1415 read 0x5993b65 thru 0x599db42 (0x9fde bytes) 2024-09-21T14:21:38.826 INFO:teuthology.orchestra.run.smithi073.stdout:1417 read 0x7011dd5 thru 0x7017a1f (0x5c4b bytes) 2024-09-21T14:21:38.827 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:38.828 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:38.828 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:38.828 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:38.830 INFO:teuthology.orchestra.run.smithi073.stdout:1418 write 0xb1c87f3 thru 0xb1c8c1f (0x42d bytes) 2024-09-21T14:21:38.833 INFO:teuthology.orchestra.run.smithi073.stdout:1423 read 0x41d5863 thru 0x41df05d (0x97fb bytes) 2024-09-21T14:21:38.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:38.839 INFO:teuthology.orchestra.run.smithi073.stdout:1424 trunc from 0xb1c8c20 to 0x667ec96 2024-09-21T14:21:38.844 INFO:teuthology.orchestra.run.smithi073.stdout:1425 write 0x384c953 thru 0x384ff3d (0x35eb bytes) 2024-09-21T14:21:38.846 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.858 INFO:teuthology.orchestra.run.smithi073.stdout:1428 write 0xcf9339f thru 0xcfa10dd (0xdd3f bytes) 2024-09-21T14:21:38.861 INFO:teuthology.orchestra.run.smithi073.stdout:1429 write 0x164b8ea thru 0x16528a5 (0x6fbc bytes) 2024-09-21T14:21:38.864 INFO:teuthology.orchestra.run.smithi073.stdout:1430 read 0x2860225 thru 0x28662b6 (0x6092 bytes) 2024-09-21T14:21:38.865 INFO:teuthology.orchestra.run.smithi073.stdout:1431 write 0x8947038 thru 0x8954dd1 (0xdd9a bytes) 2024-09-21T14:21:38.868 INFO:teuthology.orchestra.run.smithi073.stdout:1433 writesame 0x3185c7d thru 0x3192358 data_size 0x59(0xc6dc bytes) 2024-09-21T14:21:38.869 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:1435 trunc from 0xcfa10de to 0xc1ed80c 2024-09-21T14:21:38.876 INFO:teuthology.orchestra.run.smithi073.stdout:1436 write 0x75b886f thru 0x75c56f2 (0xce84 bytes) 2024-09-21T14:21:38.878 INFO:teuthology.orchestra.run.smithi073.stdout:1439 read 0x897a9f2 thru 0x897d0b4 (0x26c3 bytes) 2024-09-21T14:21:38.878 INFO:teuthology.orchestra.run.smithi073.stdout:1440 read 0x4ac10a5 thru 0x4ac1791 (0x6ed bytes) 2024-09-21T14:21:38.878 INFO:teuthology.orchestra.run.smithi073.stdout:1443 read 0x6048f3c thru 0x6049188 (0x24d bytes) 2024-09-21T14:21:38.885 INFO:teuthology.orchestra.run.smithi073.stdout:1444 trunc from 0xc1ed80c to 0x6b5c706 2024-09-21T14:21:38.892 INFO:teuthology.orchestra.run.smithi073.stdout:1445 read 0xd2db50 thru 0xd3349f (0x5950 bytes) 2024-09-21T14:21:38.895 INFO:teuthology.orchestra.run.smithi073.stdout:1447 trunc from 0x6b5c706 to 0x3c68371 2024-09-21T14:21:38.907 INFO:teuthology.orchestra.run.smithi073.stdout:1448 write 0x267741f thru 0x267ecf1 (0x78d3 bytes) 2024-09-21T14:21:38.907 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:38.908+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:38.917 INFO:teuthology.orchestra.run.smithi073.stdout:1449 writesame 0x4ed04c8 thru 0x4edceba data_size 0x13(0xc9f3 bytes) 2024-09-21T14:21:38.917 INFO:teuthology.orchestra.run.smithi073.stdout:1450 trunc from 0x4edcebb to 0x40fe5fd 2024-09-21T14:21:38.938 INFO:teuthology.orchestra.run.smithi073.stdout:1453 write 0xbe60627 thru 0xbe6e047 (0xda21 bytes) 2024-09-21T14:21:38.941 INFO:teuthology.orchestra.run.smithi073.stdout:1454 read 0xa4983b6 thru 0xa49e9b5 (0x6600 bytes) 2024-09-21T14:21:38.954 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:38.955 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:38.955 INFO:teuthology.orchestra.run.smithi073.stdout:1455 trunc from 0xbe6e048 to 0xa52c39 2024-09-21T14:21:38.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:38.990 INFO:teuthology.orchestra.run.smithi073.stdout:1456 write 0x73df4cd thru 0x73ead97 (0xb8cb bytes) 2024-09-21T14:21:38.992 INFO:teuthology.orchestra.run.smithi073.stdout:1457 read 0x3401a37 thru 0x341094e (0xef18 bytes) 2024-09-21T14:21:38.992 INFO:teuthology.orchestra.run.smithi073.stdout:1458 read 0x65b2d91 thru 0x65bd55b (0xa7cb bytes) 2024-09-21T14:21:39.004 INFO:teuthology.orchestra.run.smithi073.stdout:1461 writesame 0xdd7b9dd thru 0xdd8370f data_size 0x1(0x7d33 bytes) 2024-09-21T14:21:39.004 INFO:teuthology.orchestra.run.smithi073.stdout:1462 read 0x42273bb thru 0x4230621 (0x9267 bytes) 2024-09-21T14:21:39.005 INFO:teuthology.orchestra.run.smithi073.stdout:1463 writesame 0x3f1276 thru 0x3fdc7b data_size 0x13(0xca06 bytes) 2024-09-21T14:21:39.006 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.006 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.007 INFO:teuthology.orchestra.run.smithi073.stdout:1467 writesame 0xaf0d679 thru 0xaf1a062 data_size 0xf(0xc9ea bytes) 2024-09-21T14:21:39.009 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:39.009+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:39.010 INFO:teuthology.orchestra.run.smithi073.stdout:1468 read 0xa5f12c2 thru 0xa5f540a (0x4149 bytes) 2024-09-21T14:21:39.020 INFO:teuthology.orchestra.run.smithi073.stdout:1469 trunc from 0xdd83710 to 0x52170dd 2024-09-21T14:21:39.029 INFO:teuthology.orchestra.run.smithi073.stdout:1471 read 0x3d7f7c6 thru 0x3d805bc (0xdf7 bytes) 2024-09-21T14:21:39.030 INFO:teuthology.orchestra.run.smithi073.stdout:1473 read 0x4e844a2 thru 0x4e8f782 (0xb2e1 bytes) 2024-09-21T14:21:39.030 INFO:teuthology.orchestra.run.smithi073.stdout:1474 punch from 0x91e004 to 0x9218ff, (0x38fb bytes) 2024-09-21T14:21:39.035 INFO:teuthology.orchestra.run.smithi073.stdout:1475 write 0x7f2fb0f thru 0x7f32b8e (0x3080 bytes) 2024-09-21T14:21:39.047 INFO:teuthology.orchestra.run.smithi073.stdout:1476 write 0xc0101fd thru 0xc01db28 (0xd92c bytes) 2024-09-21T14:21:39.050 INFO:teuthology.orchestra.run.smithi073.stdout:1477 write 0xa9eac18 thru 0xa9f71b3 (0xc59c bytes) 2024-09-21T14:21:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:1479 punch from 0xb119cd2 to 0xb126452, (0xc780 bytes) 2024-09-21T14:21:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:1483 read 0x4411524 thru 0x441b901 (0xa3de bytes) 2024-09-21T14:21:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:1484 write 0xab62d3d thru 0xab6ead6 (0xbd9a bytes) 2024-09-21T14:21:39.053 INFO:teuthology.orchestra.run.smithi073.stdout:1485 writesame 0x29c2f85 thru 0x29c702c data_size 0x4(0x40a8 bytes) 2024-09-21T14:21:39.056 INFO:teuthology.orchestra.run.smithi073.stdout:1486 write 0x161ee37 thru 0x16233ae (0x4578 bytes) 2024-09-21T14:21:39.059 INFO:teuthology.orchestra.run.smithi073.stdout:1487 read 0xa6bc0c8 thru 0xa6c6670 (0xa5a9 bytes) 2024-09-21T14:21:39.068 INFO:teuthology.orchestra.run.smithi073.stdout:1488 trunc from 0xc01db29 to 0x466b23c 2024-09-21T14:21:39.080 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.080 INFO:teuthology.orchestra.run.smithi073.stdout:1490 read 0x433c89c thru 0x4347be0 (0xb345 bytes) 2024-09-21T14:21:39.092 INFO:teuthology.orchestra.run.smithi073.stdout:1491 write 0x9fb0864 thru 0x9fb1eb7 (0x1654 bytes) 2024-09-21T14:21:39.094 INFO:teuthology.orchestra.run.smithi073.stdout:1492 write 0x86eea26 thru 0x86fe668 (0xfc43 bytes) 2024-09-21T14:21:39.098 INFO:teuthology.orchestra.run.smithi073.stdout:1493 trunc from 0x9fb1eb8 to 0x855abf2 2024-09-21T14:21:39.104 INFO:teuthology.orchestra.run.smithi073.stdout:1496 write 0x57b04aa thru 0x57be191 (0xdce8 bytes) 2024-09-21T14:21:39.106 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11320 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7938206060000006, 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041544266000000003, 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11320 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7999999999999993e-07 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311102+0000", 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1620000000000001e-06 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311110+0000", 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0819999999999999e-06 2024-09-21T14:21:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352476+0000", 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041366078000000001 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352554+0000", 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7965000000000003e-05 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352641+0000", 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7194999999999994e-05 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11321 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7938053670000009, 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041652646000000002, 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11321 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311116+0000", 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311119+0000", 2024-09-21T14:21:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.45e-06 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311122+0000", 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3690000000000001e-06 2024-09-21T14:21:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352670+0000", 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041548359 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352714+0000", 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3461000000000003e-05 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352765+0000", 2024-09-21T14:21:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0911000000000001e-05 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11323 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 22528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7937934680000005, 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041853842000000002, 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11323 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:39.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311125+0000", 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.051e-06 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311128+0000", 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.886e-06 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311130+0000", 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.34e-06 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352794+0000", 2024-09-21T14:21:39.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041663797000000002 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352871+0000", 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7476000000000002e-05 2024-09-21T14:21:39.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352978+0000", 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106492 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11324 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7937853659999998, 2024-09-21T14:21:39.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042007186000000002, 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.137 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11324 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:39.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311133+0000", 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559999999999999e-06 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311136+0000", 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7829999999999999e-06 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311138+0000", 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.362e-06 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353020+0000", 2024-09-21T14:21:39.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041881717999999998 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353078+0000", 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8344999999999998e-05 2024-09-21T14:21:39.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353139+0000", 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1124000000000006e-05 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11325 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 59392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:39.146 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7937774300000005, 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042198257000000003, 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11325 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311141+0000", 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.017e-06 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311143+0000", 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7719999999999999e-06 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311146+0000", 2024-09-21T14:21:39.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.362e-06 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353178+0000", 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042032067999999999 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353257+0000", 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9500999999999997e-05 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353338+0000", 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0745e-05 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11326 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7937685630000004, 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042324452999999998, 2024-09-21T14:21:39.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11326 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.157 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311150+0000", 2024-09-21T14:21:39.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0210000000000001e-06 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311152+0000", 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7750000000000001e-06 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311155+0000", 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3719999999999999e-06 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353368+0000", 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042213186999999999 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353419+0000", 2024-09-21T14:21:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1542000000000002e-05 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353473+0000", 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3761000000000002e-05 2024-09-21T14:21:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11328 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7937606860000006, 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042440522000000001, 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11328 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311156+0000", 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8599999999999996e-07 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311160+0000", 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8229999999999999e-06 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311163+0000", 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353496+0000", 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042333000000000003 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353546+0000", 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0818e-05 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353597+0000", 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0796e-05 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11330 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 92160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7936360439999994, 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042450188999999999, 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11330 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311164+0000", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9998832 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311284+0000", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119777 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311294+0000", 2024-09-21T14:21:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0342e-05 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311300+0000", 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1580000000000004e-06 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353629+0000", 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042329929000000002 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353678+0000", 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.829e-05 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353731+0000", 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3687999999999998e-05 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11341 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 182272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7917739019999992, 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041454537999999999, 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11341 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313145+0000", 2024-09-21T14:21:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2730000000000001e-06 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313151+0000", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313156+0000", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.724e-06 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354503+0000", 2024-09-21T14:21:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041346489 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354549+0000", 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5985000000000001e-05 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354598+0000", 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9375000000000001e-05 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11342 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7917567719999994, 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041568147999999999, 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11342 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313160+0000", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313162+0000", 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313169+0000", 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.528e-06 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313173+0000", 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3959999999999999e-06 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354626+0000", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041452799999999998 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354676+0000", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9543999999999999e-05 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354729+0000", 2024-09-21T14:21:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3103000000000003e-05 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11344 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7917399599999992, 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041669309000000002, 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11344 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313179+0000", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.075e-06 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313185+0000", 2024-09-21T14:21:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7769999999999999e-06 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313190+0000", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8520000000000003e-06 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354750+0000", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041560198999999999 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354798+0000", 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7976000000000003e-05 2024-09-21T14:21:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354847+0000", 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8797000000000001e-05 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11345 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7917227679999996, 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041777117000000003, 2024-09-21T14:21:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11345 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313194+0000", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313196+0000", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1780000000000002e-06 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313202+0000", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1190000000000002e-06 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313207+0000", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4390000000000002e-06 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354874+0000", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041666789000000003 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354922+0000", 2024-09-21T14:21:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8096000000000002e-05 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354972+0000", 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9901999999999998e-05 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11347 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7917064650000007, 2024-09-21T14:21:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041891943000000001, 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11347 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313213+0000", 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0389999999999999e-06 2024-09-21T14:21:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313221+0000", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8149999999999995e-06 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313231+0000", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2199999999999998e-06 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355002+0000", 2024-09-21T14:21:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041771097 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355051+0000", 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9821000000000003e-05 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355103+0000", 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1292999999999998e-05 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11348 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 276480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7916591579999999, 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041973968, 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11348 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313261+0000", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9119999999999998e-06 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313268+0000", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8520000000000004e-06 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313273+0000", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8960000000000002e-06 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355130+0000", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041856972999999999 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355180+0000", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9264999999999997e-05 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355232+0000", 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2617999999999997e-05 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11366 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 436224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7896992090000001, 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041505316, 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11366 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315220+0000", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.024e-06 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315226+0000", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8640000000000001e-06 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315231+0000", 2024-09-21T14:21:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.882e-06 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356622+0000", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041391773999999999 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356674+0000", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1255999999999999e-05 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356723+0000", 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9867000000000002e-05 2024-09-21T14:21:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11380 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7878128340000004, 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041491635999999998, 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11380 2024-09-21T14:21:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317105+0000", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.173e-06 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317108+0000", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8770000000000002e-06 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317111+0000", 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4830000000000002e-06 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358429+0000", 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041318393000000002 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358514+0000", 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5115000000000001e-05 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358596+0000", 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1808e-05 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11381 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 129024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7878044390000003, 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041612616999999998, 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11381 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317114+0000", 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.144e-06 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317117+0000", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8080000000000001e-06 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317119+0000", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358624+0000", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041504486 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358670+0000", 2024-09-21T14:21:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6217999999999997e-05 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358725+0000", 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5733999999999999e-05 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11383 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7877960819999998, 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041761832999999998, 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11383 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317123+0000", 2024-09-21T14:21:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300000000000001e-06 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317127+0000", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8300000000000003e-06 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317130+0000", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5229999999999998e-06 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358759+0000", 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041629475999999999 2024-09-21T14:21:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358831+0000", 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1283000000000002e-05 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358883+0000", 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2349999999999999e-05 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11385 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.78776343, 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041857556999999997, 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11385 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317153+0000", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317157+0000", 2024-09-21T14:21:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.614e-06 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317164+0000", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9720000000000003e-06 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317168+0000", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4870000000000004e-06 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358912+0000", 2024-09-21T14:21:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041743495999999998 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358952+0000", 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0462e-05 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359011+0000", 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9110000000000002e-05 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11386 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 186368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.7877455480000002, 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041945596000000002, 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11386 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317171+0000", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317173+0000", 2024-09-21T14:21:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4610000000000001e-06 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317176+0000", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.337e-06 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317179+0000", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.605e-06 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359023+0000", 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041844488999999999 2024-09-21T14:21:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359069+0000", 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5667000000000002e-05 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359117+0000", 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8371000000000002e-05 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:39.226 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1498 read 0x766fe10 thru 0x7677db8 (0x7fa9 bytes) 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1499 read 0x3c42a02 thru 0x3c43793 (0xd92 bytes) 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1501 punch from 0x6c245ec to 0x6c24cce, (0x6e2 bytes) 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1506 write 0x6cd5ed7 thru 0x6cde74f (0x8879 bytes) 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1508 punch from 0x39c9a3 to 0x3a40a0, (0x76fd bytes) 2024-09-21T14:21:39.228 INFO:teuthology.orchestra.run.smithi073.stdout:1512 write 0x45c5616 thru 0x45c9700 (0x40eb bytes) 2024-09-21T14:21:39.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1513 read 0xa43356 thru 0xa529de (0xf689 bytes) 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1514 write 0x2e45c4d thru 0x2e52afe (0xceb2 bytes) 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1515 read 0x524e04d thru 0x52505dc (0x2590 bytes) 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1519 punch from 0x5a7bafa to 0x5a8516a, (0x9670 bytes) 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1520 punch from 0x3d8fb26 to 0x3d9c064, (0xc53e bytes) 2024-09-21T14:21:39.230 INFO:teuthology.orchestra.run.smithi073.stdout:1523 read 0x2ab06d6 thru 0x2aba4dc (0x9e07 bytes) 2024-09-21T14:21:39.231 INFO:teuthology.orchestra.run.smithi073.stdout:1525 punch from 0x2f0d22 to 0x2f1223, (0x501 bytes) 2024-09-21T14:21:39.231 INFO:teuthology.orchestra.run.smithi073.stdout:1526 clone 12 order 20 su 65536 sc 14 2024-09-21T14:21:39.233 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:39.228+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:39.254 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:21:39.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T14:21:39.320 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone11 from 0x855abf2 (overlap 0x2aaa28) to 0x2665f0 2024-09-21T14:21:39.328 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:39.329+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:39.365 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:39.365 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:39.365 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:39.365 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:39.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:39.380 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T14:21:39.429 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:39.429+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:39.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:39.501 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:39.501 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:39.501 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:39.501 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:39.501 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:39.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:39.552 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:39.552+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:39.615 INFO:teuthology.orchestra.run.smithi073.stdout:1527 punch from 0x3f2df28 to 0x3f334a1, (0x5579 bytes) 2024-09-21T14:21:39.616 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:39.616 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:39.627 INFO:teuthology.orchestra.run.smithi073.stdout:1528 trunc from 0x855abf2 to 0x1f632e5 2024-09-21T14:21:39.631 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:39.631+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:39.645 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:39.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:39.647 INFO:teuthology.orchestra.run.smithi073.stdout:1529 read 0x15fc570 thru 0x15fdc65 (0x16f6 bytes) 2024-09-21T14:21:39.648 INFO:teuthology.orchestra.run.smithi073.stdout:1530 punch from 0x8af79f to 0x8b2482, (0x2ce3 bytes) 2024-09-21T14:21:39.648 INFO:teuthology.orchestra.run.smithi073.stdout:1531 read 0x7c6c2 thru 0x888e9 (0xc228 bytes) 2024-09-21T14:21:39.661 INFO:teuthology.orchestra.run.smithi073.stdout:1532 trunc from 0x1f632e5 to 0xcd18a20 2024-09-21T14:21:39.663 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:39.663 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:39.666 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.670 INFO:teuthology.orchestra.run.smithi073.stdout:1534 trunc from 0xcd18a20 to 0x9834763 2024-09-21T14:21:39.679 INFO:teuthology.orchestra.run.smithi073.stdout:1535 write 0x9b1d2ad thru 0x9b26d85 (0x9ad9 bytes) 2024-09-21T14:21:39.681 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:39.681 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:39.683 INFO:teuthology.orchestra.run.smithi073.stdout:1536 punch from 0x8eee238 to 0x8efd5ef, (0xf3b7 bytes) 2024-09-21T14:21:39.683 INFO:teuthology.orchestra.run.smithi073.stdout:1538 trunc from 0x9b26d86 to 0x9192eaa 2024-09-21T14:21:39.689 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:39.689 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:39.697 INFO:teuthology.orchestra.run.smithi073.stdout:1539 trunc from 0x9192eaa to 0xd502426 2024-09-21T14:21:39.699 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:39.699 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:39.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:39.702 INFO:teuthology.orchestra.run.smithi073.stdout:1540 read 0x813bc45 thru 0x8143739 (0x7af5 bytes) 2024-09-21T14:21:39.703 INFO:teuthology.orchestra.run.smithi073.stdout:1542 punch from 0xb1521e1 to 0xb160b22, (0xe941 bytes) 2024-09-21T14:21:39.703 INFO:teuthology.orchestra.run.smithi073.stdout:1543 writesame 0x4c78e02 thru 0x4c86614 data_size 0x73(0xd813 bytes) 2024-09-21T14:21:39.716 INFO:teuthology.orchestra.run.smithi073.stdout:1544 writesame 0xe5ac996 thru 0xe5bc0d8 data_size 0x1(0xf743 bytes) 2024-09-21T14:21:39.716 INFO:teuthology.orchestra.run.smithi073.stdout:1545 write 0x32d5958 thru 0x32dff70 (0xa619 bytes) 2024-09-21T14:21:39.717 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:39.719 INFO:teuthology.orchestra.run.smithi073.stdout:1546 read 0x8129bbb thru 0x812f549 (0x598f bytes) 2024-09-21T14:21:39.719 INFO:teuthology.orchestra.run.smithi073.stdout:1547 read 0x349121b thru 0x3492842 (0x1628 bytes) 2024-09-21T14:21:39.719 INFO:teuthology.orchestra.run.smithi073.stdout:1548 writesame 0x1353d96 thru 0x135b2eb data_size 0x17(0x7556 bytes) 2024-09-21T14:21:39.724 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.724 INFO:teuthology.orchestra.run.smithi073.stdout:1551 read 0xc08ab30 thru 0xc090f8d (0x645e bytes) 2024-09-21T14:21:39.725 INFO:teuthology.orchestra.run.smithi073.stdout:1553 writesame 0xdab10 thru 0xe7958 data_size 0x1d(0xce49 bytes) 2024-09-21T14:21:39.732 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:39.732+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:39.733 INFO:teuthology.orchestra.run.smithi073.stdout:1557 punch from 0x9aef963 to 0x9af750e, (0x7bab bytes) 2024-09-21T14:21:39.733 INFO:teuthology.orchestra.run.smithi073.stdout:1558 writesame 0xb319060 thru 0xb31c9f6 data_size 0x1(0x3997 bytes) 2024-09-21T14:21:39.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:39.740 INFO:teuthology.orchestra.run.smithi073.stdout:1559 write 0x2a3aba3 thru 0x2a497a1 (0xebff bytes) 2024-09-21T14:21:39.744 INFO:teuthology.orchestra.run.smithi073.stdout:1561 write 0xa22856f thru 0xa236542 (0xdfd4 bytes) 2024-09-21T14:21:39.749 INFO:teuthology.orchestra.run.smithi073.stdout:1562 write 0x2835b58 thru 0x284324e (0xd6f7 bytes) 2024-09-21T14:21:39.750 INFO:teuthology.orchestra.run.smithi073.stdout:1565 read 0x306166a thru 0x306fc83 (0xe61a bytes) 2024-09-21T14:21:39.751 INFO:teuthology.orchestra.run.smithi073.stdout:1566 trunc from 0xe5bc0d9 to 0xe3701d7 2024-09-21T14:21:39.756 INFO:teuthology.orchestra.run.smithi073.stdout:1567 read 0x3d820c3 thru 0x3d908f9 (0xe837 bytes) 2024-09-21T14:21:39.759 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:39.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:39.759 INFO:teuthology.orchestra.run.smithi073.stdout:1568 trunc from 0xe3701d7 to 0xb321d4b 2024-09-21T14:21:39.765 INFO:teuthology.orchestra.run.smithi073.stdout:1569 read 0x6e5a9 thru 0x6ffc1 (0x1a19 bytes) 2024-09-21T14:21:39.769 INFO:teuthology.orchestra.run.smithi073.stdout:1571 trunc from 0xb321d4b to 0xd75a19f 2024-09-21T14:21:39.774 INFO:teuthology.orchestra.run.smithi073.stdout:1572 write 0x8457b98 thru 0x8465907 (0xdd70 bytes) 2024-09-21T14:21:39.779 INFO:teuthology.orchestra.run.smithi073.stdout:1573 write 0x34915e9 thru 0x349a4d1 (0x8ee9 bytes) 2024-09-21T14:21:39.781 INFO:teuthology.orchestra.run.smithi073.stdout:1574 write 0x4741aa2 thru 0x474591b (0x3e7a bytes) 2024-09-21T14:21:39.782 INFO:teuthology.orchestra.run.smithi073.stdout:1576 read 0x6d7db15 thru 0x6d88cb4 (0xb1a0 bytes) 2024-09-21T14:21:39.782 INFO:teuthology.orchestra.run.smithi073.stdout:1577 write 0x85cabdb thru 0x85d28fb (0x7d21 bytes) 2024-09-21T14:21:39.786 INFO:teuthology.orchestra.run.smithi073.stdout:1578 write 0xafbfef4 thru 0xafcd489 (0xd596 bytes) 2024-09-21T14:21:39.792 INFO:teuthology.orchestra.run.smithi073.stdout:1579 write 0x60eee24 thru 0x60f1861 (0x2a3e bytes) 2024-09-21T14:21:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:1580 read 0x4c15ef7 thru 0x4c1c94b (0x6a55 bytes) 2024-09-21T14:21:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:1585 read 0xb7d4f08 thru 0xb7e4b68 (0xfc61 bytes) 2024-09-21T14:21:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:1587 write 0x3d88fb0 thru 0x3d94c32 (0xbc83 bytes) 2024-09-21T14:21:39.811 INFO:teuthology.orchestra.run.smithi073.stdout:1588 trunc from 0xd75a19f to 0x77a8db5 2024-09-21T14:21:39.821 INFO:teuthology.orchestra.run.smithi073.stdout:1592 write 0x1f10d4d thru 0x1f17716 (0x69ca bytes) 2024-09-21T14:21:39.821 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:39.821 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:39.822 INFO:teuthology.orchestra.run.smithi073.stdout:1593 read 0x2ce1e3f thru 0x2cecebd (0xb07f bytes) 2024-09-21T14:21:39.822 INFO:teuthology.orchestra.run.smithi073.stdout:1594 punch from 0x2476993 to 0x24794e7, (0x2b54 bytes) 2024-09-21T14:21:39.822 INFO:teuthology.orchestra.run.smithi073.stdout:1596 read 0x27c8267 thru 0x27d70f9 (0xee93 bytes) 2024-09-21T14:21:39.827 INFO:teuthology.orchestra.run.smithi073.stdout:1597 writesame 0x7b00b4d thru 0x7b0274b data_size 0x1(0x1bff bytes) 2024-09-21T14:21:39.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:39.835 INFO:teuthology.orchestra.run.smithi073.stdout:1598 writesame 0x9fc4ed3 thru 0x9fce123 data_size 0x1(0x9251 bytes) 2024-09-21T14:21:39.835 INFO:teuthology.orchestra.run.smithi073.stdout:1600 write 0x18d6d05 thru 0x18df81d (0x8b19 bytes) 2024-09-21T14:21:39.835 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:39.835+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:39.839 INFO:teuthology.orchestra.run.smithi073.stdout:1604 punch from 0x43e43c7 to 0x43e9f5a, (0x5b93 bytes) 2024-09-21T14:21:39.840 INFO:teuthology.orchestra.run.smithi073.stdout:1605 read 0x52521e2 thru 0x5261173 (0xef92 bytes) 2024-09-21T14:21:39.840 INFO:teuthology.orchestra.run.smithi073.stdout:1606 write 0x6cee7e2 thru 0x6cf85a5 (0x9dc4 bytes) 2024-09-21T14:21:39.843 INFO:teuthology.orchestra.run.smithi073.stdout:1607 read 0x6707ae9 thru 0x670d615 (0x5b2d bytes) 2024-09-21T14:21:39.843 INFO:teuthology.orchestra.run.smithi073.stdout:1610 read 0x3480f67 thru 0x348e4bf (0xd559 bytes) 2024-09-21T14:21:39.843 INFO:teuthology.orchestra.run.smithi073.stdout:1611 read 0x7301a62 thru 0x730824f (0x67ee bytes) 2024-09-21T14:21:39.843 INFO:teuthology.orchestra.run.smithi073.stdout:1612 punch from 0x1db0a45 to 0x1db3803, (0x2dbe bytes) 2024-09-21T14:21:39.843 INFO:teuthology.orchestra.run.smithi073.stdout:1614 write 0x43af469 thru 0x43b865a (0x91f2 bytes) 2024-09-21T14:21:39.844 INFO:teuthology.orchestra.run.smithi073.stdout:1615 read 0x52e20ff thru 0x52e77d8 (0x56da bytes) 2024-09-21T14:21:39.844 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.844 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.852 INFO:teuthology.orchestra.run.smithi073.stdout:1621 write 0xbd905e4 thru 0xbd95e7f (0x589c bytes) 2024-09-21T14:21:39.854 INFO:teuthology.orchestra.run.smithi073.stdout:1624 read 0x9db83f1 thru 0x9dc30d7 (0xace7 bytes) 2024-09-21T14:21:39.855 INFO:teuthology.orchestra.run.smithi073.stdout:1625 punch from 0x1314eeb to 0x132060a, (0xb71f bytes) 2024-09-21T14:21:39.855 INFO:teuthology.orchestra.run.smithi073.stdout:1626 writesame 0xb9a59fc thru 0xb9a734f data_size 0x2(0x1954 bytes) 2024-09-21T14:21:39.858 INFO:teuthology.orchestra.run.smithi073.stdout:1627 punch from 0x3b5e6c to 0x3c14a5, (0xb639 bytes) 2024-09-21T14:21:39.858 INFO:teuthology.orchestra.run.smithi073.stdout:1630 write 0x692af0c thru 0x69342b4 (0x93a9 bytes) 2024-09-21T14:21:39.862 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.868 INFO:teuthology.orchestra.run.smithi073.stdout:1634 writesame 0xd0c5537 thru 0xd0cb7fe data_size 0x1d(0x62c8 bytes) 2024-09-21T14:21:39.878 INFO:teuthology.orchestra.run.smithi073.stdout:1635 trunc from 0xd0cb7ff to 0x5bf5721 2024-09-21T14:21:39.906 INFO:teuthology.orchestra.run.smithi073.stdout:1638 write 0xe8dd3d4 thru 0xe8df9a2 (0x25cf bytes) 2024-09-21T14:21:39.909 INFO:teuthology.orchestra.run.smithi073.stdout:1640 punch from 0xcf20d29 to 0xcf2dfc4, (0xd29b bytes) 2024-09-21T14:21:39.909 INFO:teuthology.orchestra.run.smithi073.stdout:1642 read 0x7a73961 thru 0x7a76ed1 (0x3571 bytes) 2024-09-21T14:21:39.909 INFO:teuthology.orchestra.run.smithi073.stdout:1643 punch from 0xb3449ea to 0xb344f7e, (0x594 bytes) 2024-09-21T14:21:39.909 INFO:teuthology.orchestra.run.smithi073.stdout:1644 write 0xddcf9d2 thru 0xddd310a (0x3739 bytes) 2024-09-21T14:21:39.913 INFO:teuthology.orchestra.run.smithi073.stdout:1647 writesame 0x609cb93 thru 0x60a0efc data_size 0x1(0x436a bytes) 2024-09-21T14:21:39.922 INFO:teuthology.orchestra.run.smithi073.stdout:1649 read 0x99b860a thru 0x99c1803 (0x91fa bytes) 2024-09-21T14:21:39.922 INFO:teuthology.orchestra.run.smithi073.stdout:1653 read 0x307b350 thru 0x30856de (0xa38f bytes) 2024-09-21T14:21:39.922 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.922 INFO:teuthology.orchestra.run.smithi073.stdout:1655 write 0x2b64c74 thru 0x2b6e9f5 (0x9d82 bytes) 2024-09-21T14:21:39.925 INFO:teuthology.orchestra.run.smithi073.stdout:1656 writesame 0xa7b2c64 thru 0xa7bea38 data_size 0x3(0xbdd5 bytes) 2024-09-21T14:21:39.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:39.933+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:39.935 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.935 INFO:teuthology.orchestra.run.smithi073.stdout:1658 read 0xc13246f thru 0xc13c537 (0xa0c9 bytes) 2024-09-21T14:21:39.935 INFO:teuthology.orchestra.run.smithi073.stdout:1659 punch from 0x13a2b14 to 0x13b1d09, (0xf1f5 bytes) 2024-09-21T14:21:39.935 INFO:teuthology.orchestra.run.smithi073.stdout:1660 write 0x45f6fd0 thru 0x45f784f (0x880 bytes) 2024-09-21T14:21:39.953 INFO:teuthology.orchestra.run.smithi073.stdout:1661 trunc from 0xe8df9a3 to 0x15ae35c 2024-09-21T14:21:39.962 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:39.962 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:39.973 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:39.997 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.997 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:39.997 INFO:teuthology.orchestra.run.smithi073.stdout:1664 read 0x1574da0 thru 0x1579c15 (0x4e76 bytes) 2024-09-21T14:21:39.997 INFO:teuthology.orchestra.run.smithi073.stdout:1666 writesame 0x6e1447 thru 0x6efa05 data_size 0xf(0xe5bf bytes) 2024-09-21T14:21:40.007 INFO:teuthology.orchestra.run.smithi073.stdout:1668 writesame 0x1681e09 thru 0x168c0f3 data_size 0x1(0xa2eb bytes) 2024-09-21T14:21:40.014 INFO:teuthology.orchestra.run.smithi073.stdout:1670 trunc from 0x168c0f4 to 0x589174b 2024-09-21T14:21:40.019 INFO:teuthology.orchestra.run.smithi073.stdout:1672 punch from 0x3cd89fa to 0x3ce8294, (0xf89a bytes) 2024-09-21T14:21:40.030 INFO:teuthology.orchestra.run.smithi073.stdout:1674 writesame 0xaad8010 thru 0xaae4596 data_size 0x1(0xc587 bytes) 2024-09-21T14:21:40.030 INFO:teuthology.orchestra.run.smithi073.stdout:1675 punch from 0x92d469d to 0x92d68ec, (0x224f bytes) 2024-09-21T14:21:40.030 INFO:teuthology.orchestra.run.smithi073.stdout:1676 punch from 0x772542f to 0x7730539, (0xb10a bytes) 2024-09-21T14:21:40.030 INFO:teuthology.orchestra.run.smithi073.stdout:1677 write 0x5b2dd60 thru 0x5b3bd8b (0xe02c bytes) 2024-09-21T14:21:40.034 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:40.035+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:40.035 INFO:teuthology.orchestra.run.smithi073.stdout:1678 writesame 0x53457fd thru 0x5345cd8 data_size 0x2(0x4dc bytes) 2024-09-21T14:21:40.039 INFO:teuthology.orchestra.run.smithi073.stdout:1679 read 0x978877c thru 0x9788a28 (0x2ad bytes) 2024-09-21T14:21:40.039 INFO:teuthology.orchestra.run.smithi073.stdout:1680 writesame 0x7b927be thru 0x7b960d5 data_size 0x3f(0x3918 bytes) 2024-09-21T14:21:40.051 INFO:teuthology.orchestra.run.smithi073.stdout:1681 write 0xeb1f6b0 thru 0xeb25b28 (0x6479 bytes) 2024-09-21T14:21:40.058 INFO:teuthology.orchestra.run.smithi073.stdout:1683 read 0x8920907 thru 0x89258a1 (0x4f9b bytes) 2024-09-21T14:21:40.073 INFO:teuthology.orchestra.run.smithi073.stdout:1685 trunc from 0xeb25b29 to 0x1bfb86b 2024-09-21T14:21:40.084 INFO:teuthology.orchestra.run.smithi073.stdout:1686 read 0xd9074a thru 0xd93fec (0x38a3 bytes) 2024-09-21T14:21:40.084 INFO:teuthology.orchestra.run.smithi073.stdout:1687 read 0x47928c thru 0x482e2d (0x9ba2 bytes) 2024-09-21T14:21:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:1688 write 0x6ec5159 thru 0x6ecb0fa (0x5fa2 bytes) 2024-09-21T14:21:40.112 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.112 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.112 INFO:teuthology.orchestra.run.smithi073.stdout:1689 write 0xe337b78 thru 0xe339fc3 (0x244c bytes) 2024-09-21T14:21:40.115 INFO:teuthology.orchestra.run.smithi073.stdout:1690 write 0x1ef7f6 thru 0x1fe608 (0xee13 bytes) 2024-09-21T14:21:40.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:40.122 INFO:teuthology.orchestra.run.smithi073.stdout:1691 writesame 0xe57d87 thru 0xe6095c data_size 0x2(0x8bd6 bytes) 2024-09-21T14:21:40.141 INFO:teuthology.orchestra.run.smithi073.stdout:1692 trunc from 0xe339fc4 to 0x32b8665 2024-09-21T14:21:40.142 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:40.142+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:40.151 INFO:teuthology.orchestra.run.smithi073.stdout:1693 read 0x5e0c91 thru 0x5ea712 (0x9a82 bytes) 2024-09-21T14:21:40.151 INFO:teuthology.orchestra.run.smithi073.stdout:1694 read 0x31b0f23 thru 0x31b1ec1 (0xf9f bytes) 2024-09-21T14:21:40.151 INFO:teuthology.orchestra.run.smithi073.stdout:1695 read 0x24fa3b9 thru 0x25054b8 (0xb100 bytes) 2024-09-21T14:21:40.151 INFO:teuthology.orchestra.run.smithi073.stdout:1696 read 0x2c3e413 thru 0x2c4af8e (0xcb7c bytes) 2024-09-21T14:21:40.162 INFO:teuthology.orchestra.run.smithi073.stdout:1697 trunc from 0x32b8665 to 0xbe83326 2024-09-21T14:21:40.167 INFO:teuthology.orchestra.run.smithi073.stdout:1698 punch from 0xb25e748 to 0xb26d32b, (0xebe3 bytes) 2024-09-21T14:21:40.167 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.180 INFO:teuthology.orchestra.run.smithi073.stdout:1700 trunc from 0xbe83326 to 0x139867d 2024-09-21T14:21:40.190 INFO:teuthology.orchestra.run.smithi073.stdout:1702 read 0xbb64a1 thru 0xbb9717 (0x3277 bytes) 2024-09-21T14:21:40.191 INFO:teuthology.orchestra.run.smithi073.stdout:1703 trunc from 0x139867d to 0x193e638 2024-09-21T14:21:40.195 INFO:teuthology.orchestra.run.smithi073.stdout:1705 punch from 0x18e59d4 to 0x18e7c02, (0x222e bytes) 2024-09-21T14:21:40.204 INFO:teuthology.orchestra.run.smithi073.stdout:1706 writesame 0x4afa3d8 thru 0x4b00f22 data_size 0x1(0x6b4b bytes) 2024-09-21T14:21:40.204 INFO:teuthology.orchestra.run.smithi073.stdout:1707 read 0xca0d6 thru 0xd031e (0x6249 bytes) 2024-09-21T14:21:40.204 INFO:teuthology.orchestra.run.smithi073.stdout:1708 write 0x6ff181 thru 0x7064b7 (0x7337 bytes) 2024-09-21T14:21:40.205 INFO:teuthology.orchestra.run.smithi073.stdout:1710 read 0x494b2f3 thru 0x4955605 (0xa313 bytes) 2024-09-21T14:21:40.206 INFO:teuthology.orchestra.run.smithi073.stdout:1712 read 0x33e6c94 thru 0x33ecf1e (0x628b bytes) 2024-09-21T14:21:40.206 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.206 INFO:teuthology.orchestra.run.smithi073.stdout:1714 read 0x1cac950 thru 0x1cad477 (0xb28 bytes) 2024-09-21T14:21:40.211 INFO:teuthology.orchestra.run.smithi073.stdout:1715 trunc from 0x4b00f23 to 0x9b6961d 2024-09-21T14:21:40.216 INFO:teuthology.orchestra.run.smithi073.stdout:1716 punch from 0x71a9fa6 to 0x71b1378, (0x73d2 bytes) 2024-09-21T14:21:40.217 INFO:teuthology.orchestra.run.smithi073.stdout:1717 write 0x570fa26 thru 0x570fb85 (0x160 bytes) 2024-09-21T14:21:40.219 INFO:teuthology.orchestra.run.smithi073.stdout:1719 read 0x1176e6c thru 0x1185dc8 (0xef5d bytes) 2024-09-21T14:21:40.219 INFO:teuthology.orchestra.run.smithi073.stdout:1720 read 0x23375e1 thru 0x2341c44 (0xa664 bytes) 2024-09-21T14:21:40.219 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.220 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.227 INFO:teuthology.orchestra.run.smithi073.stdout:1726 writesame 0xc55ef8b thru 0xc56da39 data_size 0x1(0xeaaf bytes) 2024-09-21T14:21:40.227 INFO:teuthology.orchestra.run.smithi073.stdout:1727 write 0x1dd36b3 thru 0x1dd3990 (0x2de bytes) 2024-09-21T14:21:40.230 INFO:teuthology.orchestra.run.smithi073.stdout:1728 write 0xc1513f5 thru 0xc15e756 (0xd362 bytes) 2024-09-21T14:21:40.233 INFO:teuthology.orchestra.run.smithi073.stdout:1730 read 0x5e2b141 thru 0x5e329c7 (0x7887 bytes) 2024-09-21T14:21:40.233 INFO:teuthology.orchestra.run.smithi073.stdout:1731 write 0x239b8e4 thru 0x23a03c6 (0x4ae3 bytes) 2024-09-21T14:21:40.236 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:40.236+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:40.236 INFO:teuthology.orchestra.run.smithi073.stdout:1733 writesame 0x33ee134 thru 0x33f22aa data_size 0x1(0x4177 bytes) 2024-09-21T14:21:40.243 INFO:teuthology.orchestra.run.smithi073.stdout:1735 writesame 0x5a37728 thru 0x5a45c53 data_size 0x6(0xe52c bytes) 2024-09-21T14:21:40.249 INFO:teuthology.orchestra.run.smithi073.stdout:1736 read 0x8428f56 thru 0x842fe01 (0x6eac bytes) 2024-09-21T14:21:40.254 INFO:teuthology.orchestra.run.smithi073.stdout:1737 trunc from 0xc56da3a to 0xa06bede 2024-09-21T14:21:40.258 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.258 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.263 INFO:teuthology.orchestra.run.smithi073.stdout:1738 write 0x12e923e thru 0x12ece5f (0x3c22 bytes) 2024-09-21T14:21:40.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:21:40.265 INFO:teuthology.orchestra.run.smithi073.stdout:1739 punch from 0x5dc6f80 to 0x5dcaaf6, (0x3b76 bytes) 2024-09-21T14:21:40.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:40.272 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T14:21:40.275 INFO:teuthology.orchestra.run.smithi073.stdout:1740 trunc from 0xa06bede to 0x1c8835e 2024-09-21T14:21:40.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T14:21:40.294 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.315 INFO:teuthology.orchestra.run.smithi073.stdout:1742 write 0xe695419 thru 0xe6a25f3 (0xd1db bytes) 2024-09-21T14:21:40.319 INFO:teuthology.orchestra.run.smithi073.stdout:1744 write 0xd7627e6 thru 0xd766436 (0x3c51 bytes) 2024-09-21T14:21:40.323 INFO:teuthology.orchestra.run.smithi073.stdout:1746 writesame 0x640adcd thru 0x6415bb4 data_size 0x4(0xade8 bytes) 2024-09-21T14:21:40.330 INFO:teuthology.orchestra.run.smithi073.stdout:1747 read 0x71c690e thru 0x71d3e7b (0xd56e bytes) 2024-09-21T14:21:40.330 INFO:teuthology.orchestra.run.smithi073.stdout:1748 write 0xa90a352 thru 0xa90d0c3 (0x2d72 bytes) 2024-09-21T14:21:40.334 INFO:teuthology.orchestra.run.smithi073.stdout:1751 writesame 0x44f77ba thru 0x450590d data_size 0x17(0xe154 bytes) 2024-09-21T14:21:40.337 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:40.337+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:40.348 INFO:teuthology.orchestra.run.smithi073.stdout:1752 trunc from 0xe6a25f4 to 0x899afce 2024-09-21T14:21:40.360 INFO:teuthology.orchestra.run.smithi073.stdout:1754 writesame 0x3b7d618 thru 0x3b85e7d data_size 0x4f(0x8866 bytes) 2024-09-21T14:21:40.365 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.365 INFO:teuthology.orchestra.run.smithi073.stdout:1757 punch from 0x6944aea to 0x6948db0, (0x42c6 bytes) 2024-09-21T14:21:40.367 INFO:teuthology.orchestra.run.smithi073.stdout:1758 trunc from 0x899afce to 0xa598c8b 2024-09-21T14:21:40.373 INFO:teuthology.orchestra.run.smithi073.stdout:1760 punch from 0x50160ea to 0x501dbb5, (0x7acb bytes) 2024-09-21T14:21:40.373 INFO:teuthology.orchestra.run.smithi073.stdout:1762 trunc from 0xa598c8b to 0xb086deb 2024-09-21T14:21:40.391 INFO:teuthology.orchestra.run.smithi073.stdout:1763 trunc from 0xb086deb to 0x15eb513 2024-09-21T14:21:40.392 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.392 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.405 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:40.406 INFO:teuthology.orchestra.run.smithi073.stdout:1764 punch from 0x10c0207 to 0x10ce3c9, (0xe1c2 bytes) 2024-09-21T14:21:40.406 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.421 INFO:teuthology.orchestra.run.smithi073.stdout:1766 writesame 0x8a8e812 thru 0x8a90194 data_size 0x7(0x1983 bytes) 2024-09-21T14:21:40.421 INFO:teuthology.orchestra.run.smithi073.stdout:1767 read 0x9b2f75 thru 0x9c1c43 (0xeccf bytes) 2024-09-21T14:21:40.421 INFO:teuthology.orchestra.run.smithi073.stdout:1768 read 0xcdc642 thru 0xce5397 (0x8d56 bytes) 2024-09-21T14:21:40.422 INFO:teuthology.orchestra.run.smithi073.stdout:1770 read 0x658395d thru 0x65840ef (0x793 bytes) 2024-09-21T14:21:40.432 INFO:teuthology.orchestra.run.smithi073.stdout:1772 write 0xbbdb734 thru 0xbbdf59e (0x3e6b bytes) 2024-09-21T14:21:40.435 INFO:teuthology.orchestra.run.smithi073.stdout:1774 write 0x79782d7 thru 0x7983154 (0xae7e bytes) 2024-09-21T14:21:40.437 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:40.437+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:40.441 INFO:teuthology.orchestra.run.smithi073.stdout:1775 read 0x98a91c5 thru 0x98b854a (0xf386 bytes) 2024-09-21T14:21:40.448 INFO:teuthology.orchestra.run.smithi073.stdout:1776 write 0xc360fc0 thru 0xc36eb85 (0xdbc6 bytes) 2024-09-21T14:21:40.458 INFO:teuthology.orchestra.run.smithi073.stdout:1779 write 0xceb4ede thru 0xceb963b (0x475e bytes) 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:1784 punch from 0x84a2fe3 to 0x84a8a16, (0x5a33 bytes) 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:1788 read 0xaa59f0c thru 0xaa5aef3 (0xfe8 bytes) 2024-09-21T14:21:40.461 INFO:teuthology.orchestra.run.smithi073.stdout:1789 read 0x9536a89 thru 0x953dfa2 (0x751a bytes) 2024-09-21T14:21:40.464 INFO:teuthology.orchestra.run.smithi073.stdout:1790 trunc from 0xceb963c to 0x9dab306 2024-09-21T14:21:40.475 INFO:teuthology.orchestra.run.smithi073.stdout:1791 trunc from 0x9dab306 to 0xb145a97 2024-09-21T14:21:40.481 INFO:teuthology.orchestra.run.smithi073.stdout:1792 read 0x5836397 thru 0x5838485 (0x20ef bytes) 2024-09-21T14:21:40.487 INFO:teuthology.orchestra.run.smithi073.stdout:1793 trunc from 0xb145a97 to 0x659510d 2024-09-21T14:21:40.496 INFO:teuthology.orchestra.run.smithi073.stdout:1794 punch from 0x1c2f0df to 0x1c36aa0, (0x79c1 bytes) 2024-09-21T14:21:40.499 INFO:teuthology.orchestra.run.smithi073.stdout:1795 trunc from 0x659510d to 0x3b1a3db 2024-09-21T14:21:40.514 INFO:teuthology.orchestra.run.smithi073.stdout:1796 trunc from 0x3b1a3db to 0xbdd6e4e 2024-09-21T14:21:40.519 INFO:teuthology.orchestra.run.smithi073.stdout:1798 read 0x67871dd thru 0x678ad07 (0x3b2b bytes) 2024-09-21T14:21:40.519 INFO:teuthology.orchestra.run.smithi073.stdout:1801 write 0x5482451 thru 0x54862ca (0x3e7a bytes) 2024-09-21T14:21:40.522 INFO:teuthology.orchestra.run.smithi073.stdout:1802 read 0x8e25f89 thru 0x8e33628 (0xd6a0 bytes) 2024-09-21T14:21:40.522 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.522 INFO:teuthology.orchestra.run.smithi073.stdout:1807 write 0xa6be8e7 thru 0xa6c3ab8 (0x51d2 bytes) 2024-09-21T14:21:40.525 INFO:teuthology.orchestra.run.smithi073.stdout:1808 punch from 0x2d593d6 to 0x2d68992, (0xf5bc bytes) 2024-09-21T14:21:40.525 INFO:teuthology.orchestra.run.smithi073.stdout:1810 write 0xa772265 thru 0xa7789a5 (0x6741 bytes) 2024-09-21T14:21:40.529 INFO:teuthology.orchestra.run.smithi073.stdout:1811 writesame 0xa68712d thru 0xa696f15 data_size 0x2f(0xfde9 bytes) 2024-09-21T14:21:40.531 INFO:teuthology.orchestra.run.smithi073.stdout:1812 read 0xb0f0097 thru 0xb0f741c (0x7386 bytes) 2024-09-21T14:21:40.531 INFO:teuthology.orchestra.run.smithi073.stdout:1814 writesame 0x53fca2b thru 0x53fefea data_size 0x1(0x25c0 bytes) 2024-09-21T14:21:40.536 INFO:teuthology.orchestra.run.smithi073.stdout:1815 write 0x717e252 thru 0x71895c4 (0xb373 bytes) 2024-09-21T14:21:40.538 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:40.538+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:40.539 INFO:teuthology.orchestra.run.smithi073.stdout:1816 read 0x170c3cc thru 0x1715d15 (0x994a bytes) 2024-09-21T14:21:40.540 INFO:teuthology.orchestra.run.smithi073.stdout:1818 write 0x140f4c9 thru 0x141161e (0x2156 bytes) 2024-09-21T14:21:40.541 INFO:teuthology.orchestra.run.smithi073.stdout:1820 read 0x3aff3a thru 0x3b13b9 (0x1480 bytes) 2024-09-21T14:21:40.542 INFO:teuthology.orchestra.run.smithi073.stdout:1821 writesame 0xbb85492 thru 0xbb912ef data_size 0x3b(0xbe5e bytes) 2024-09-21T14:21:40.545 INFO:teuthology.orchestra.run.smithi073.stdout:1822 read 0x363659e thru 0x3637e80 (0x18e3 bytes) 2024-09-21T14:21:40.549 INFO:teuthology.orchestra.run.smithi073.stdout:1823 trunc from 0xbdd6e4e to 0x86a9e50 2024-09-21T14:21:40.559 INFO:teuthology.orchestra.run.smithi073.stdout:1824 read 0x4440e66 thru 0x4449ac4 (0x8c5f bytes) 2024-09-21T14:21:40.560 INFO:teuthology.orchestra.run.smithi073.stdout:1829 read 0x8520688 thru 0x8523d3a (0x36b3 bytes) 2024-09-21T14:21:40.560 INFO:teuthology.orchestra.run.smithi073.stdout:1830 write 0x3aa8c35 thru 0x3aaf3f7 (0x67c3 bytes) 2024-09-21T14:21:40.562 INFO:teuthology.orchestra.run.smithi073.stdout:1831 punch from 0x95c510 to 0x95eda8, (0x2898 bytes) 2024-09-21T14:21:40.570 INFO:teuthology.orchestra.run.smithi073.stdout:1832 trunc from 0x86a9e50 to 0x2228748 2024-09-21T14:21:40.575 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.575 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:40.590 INFO:teuthology.orchestra.run.smithi073.stdout:1833 writesame 0x619e319 thru 0x61a9c81 data_size 0xb(0xb969 bytes) 2024-09-21T14:21:40.602 INFO:teuthology.orchestra.run.smithi073.stdout:1835 write 0xb14f29f thru 0xb157e9b (0x8bfd bytes) 2024-09-21T14:21:40.605 INFO:teuthology.orchestra.run.smithi073.stdout:1836 read 0x3d1eca4 thru 0x3d2c289 (0xd5e6 bytes) 2024-09-21T14:21:40.608 INFO:teuthology.orchestra.run.smithi073.stdout:1838 trunc from 0xb157e9c to 0x885d1c8 2024-09-21T14:21:40.618 INFO:teuthology.orchestra.run.smithi073.stdout:1839 punch from 0x39c56e0 to 0x39c5f3f, (0x85f bytes) 2024-09-21T14:21:40.619 INFO:teuthology.orchestra.run.smithi073.stdout:1840 write 0x2c09725 thru 0x2c104cd (0x6da9 bytes) 2024-09-21T14:21:40.622 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.622 INFO:teuthology.orchestra.run.smithi073.stdout:1845 write 0x234db63 thru 0x23522f1 (0x478f bytes) 2024-09-21T14:21:40.627 INFO:teuthology.orchestra.run.smithi073.stdout:1848 trunc from 0x885d1c8 to 0x6b922be 2024-09-21T14:21:40.632 INFO:teuthology.orchestra.run.smithi073.stdout:1849 read 0x300bb4c thru 0x300e847 (0x2cfc bytes) 2024-09-21T14:21:40.632 INFO:teuthology.orchestra.run.smithi073.stdout:1850 read 0x69fd6c6 thru 0x69ff26e (0x1ba9 bytes) 2024-09-21T14:21:40.632 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.636 INFO:teuthology.orchestra.run.smithi073.stdout:1852 trunc from 0x6b922be to 0x3731487 2024-09-21T14:21:40.639 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:40.639+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:40.642 INFO:teuthology.orchestra.run.smithi073.stdout:1854 read 0x172789e thru 0x1728c0d (0x1370 bytes) 2024-09-21T14:21:40.650 INFO:teuthology.orchestra.run.smithi073.stdout:1855 writesame 0x577e170 thru 0x578a2da data_size 0x5(0xc16b bytes) 2024-09-21T14:21:40.651 INFO:teuthology.orchestra.run.smithi073.stdout:1856 read 0x389fd20 thru 0x38a37b7 (0x3a98 bytes) 2024-09-21T14:21:40.651 INFO:teuthology.orchestra.run.smithi073.stdout:1857 write 0x2c411d thru 0x2c989f (0x5783 bytes) 2024-09-21T14:21:40.652 INFO:teuthology.orchestra.run.smithi073.stdout:1859 read 0x1806c45 thru 0x180b9a5 (0x4d61 bytes) 2024-09-21T14:21:40.652 INFO:teuthology.orchestra.run.smithi073.stdout:1860 write 0x7e8388 thru 0x7f76b0 (0xf329 bytes) 2024-09-21T14:21:40.658 INFO:teuthology.orchestra.run.smithi073.stdout:1861 punch from 0x2f71e64 to 0x2f7fda5, (0xdf41 bytes) 2024-09-21T14:21:40.658 INFO:teuthology.orchestra.run.smithi073.stdout:1862 writesame 0x32f31a3 thru 0x3300558 data_size 0x9(0xd3b6 bytes) 2024-09-21T14:21:40.662 INFO:teuthology.orchestra.run.smithi073.stdout:1864 punch from 0x50b684b to 0x50bb5d1, (0x4d86 bytes) 2024-09-21T14:21:40.662 INFO:teuthology.orchestra.run.smithi073.stdout:1865 read 0xf8b62e thru 0xf93267 (0x7c3a bytes) 2024-09-21T14:21:40.662 INFO:teuthology.orchestra.run.smithi073.stdout:1866 read 0x260f465 thru 0x261d58c (0xe128 bytes) 2024-09-21T14:21:40.663 INFO:teuthology.orchestra.run.smithi073.stdout:1867 read 0x4776fe3 thru 0x4785702 (0xe720 bytes) 2024-09-21T14:21:40.663 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.675 INFO:teuthology.orchestra.run.smithi073.stdout:1870 write 0xc10331d thru 0xc111dd2 (0xeab6 bytes) 2024-09-21T14:21:40.681 INFO:teuthology.orchestra.run.smithi073.stdout:1871 read 0x2ef61bc thru 0x2efe603 (0x8448 bytes) 2024-09-21T14:21:40.690 INFO:teuthology.orchestra.run.smithi073.stdout:1872 trunc from 0xc111dd3 to 0x4b818c7 2024-09-21T14:21:40.699 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.701 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.701 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:40.713 INFO:teuthology.orchestra.run.smithi073.stdout:1874 write 0xb858b01 thru 0xb8688de (0xfdde bytes) 2024-09-21T14:21:40.718 INFO:teuthology.orchestra.run.smithi073.stdout:1875 write 0x16e10da thru 0x16e39f6 (0x291d bytes) 2024-09-21T14:21:40.720 INFO:teuthology.orchestra.run.smithi073.stdout:1876 punch from 0x63cd9b8 to 0x63da400, (0xca48 bytes) 2024-09-21T14:21:40.720 INFO:teuthology.orchestra.run.smithi073.stdout:1877 write 0x317db02 thru 0x318a9d3 (0xced2 bytes) 2024-09-21T14:21:40.733 INFO:teuthology.orchestra.run.smithi073.stdout:1878 trunc from 0xb8688df to 0x394dffd 2024-09-21T14:21:40.740 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:40.740+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:40.742 INFO:teuthology.orchestra.run.smithi073.stdout:1880 read 0x2cc8a1f thru 0x2ccf766 (0x6d48 bytes) 2024-09-21T14:21:40.749 INFO:teuthology.orchestra.run.smithi073.stdout:1881 write 0x5130770 thru 0x51365e7 (0x5e78 bytes) 2024-09-21T14:21:40.754 INFO:teuthology.orchestra.run.smithi073.stdout:1882 trunc from 0x51365e8 to 0x64849df 2024-09-21T14:21:40.759 INFO:teuthology.orchestra.run.smithi073.stdout:1883 read 0x15d6b20 thru 0x15de901 (0x7de2 bytes) 2024-09-21T14:21:40.760 INFO:teuthology.orchestra.run.smithi073.stdout:1884 write 0x14cc0cc thru 0x14d89df (0xc914 bytes) 2024-09-21T14:21:40.762 INFO:teuthology.orchestra.run.smithi073.stdout:1885 read 0x4d1b017 thru 0x4d24498 (0x9482 bytes) 2024-09-21T14:21:40.762 INFO:teuthology.orchestra.run.smithi073.stdout:1888 punch from 0x1f779e7 to 0x1f7c872, (0x4e8b bytes) 2024-09-21T14:21:40.762 INFO:teuthology.orchestra.run.smithi073.stdout:1889 write 0x1c2f563 thru 0x1c3855c (0x8ffa bytes) 2024-09-21T14:21:40.773 INFO:teuthology.orchestra.run.smithi073.stdout:1890 trunc from 0x64849df to 0x26b2267 2024-09-21T14:21:40.787 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.800 INFO:teuthology.orchestra.run.smithi073.stdout:1892 trunc from 0x26b2267 to 0xdbe9266 2024-09-21T14:21:40.805 INFO:teuthology.orchestra.run.smithi073.stdout:1893 punch from 0x1359d21 to 0x135d586, (0x3865 bytes) 2024-09-21T14:21:40.805 INFO:teuthology.orchestra.run.smithi073.stdout:1895 read 0x1316664 thru 0x13260a0 (0xfa3d bytes) 2024-09-21T14:21:40.811 INFO:teuthology.orchestra.run.smithi073.stdout:1896 writesame 0xe571c6e thru 0xe57d869 data_size 0x35(0xbbfc bytes) 2024-09-21T14:21:40.811 INFO:teuthology.orchestra.run.smithi073.stdout:1898 write 0xa8735e0 thru 0xa87fc3f (0xc660 bytes) 2024-09-21T14:21:40.815 INFO:teuthology.orchestra.run.smithi073.stdout:1899 read 0x7c7fc2e thru 0x7c83104 (0x34d7 bytes) 2024-09-21T14:21:40.815 INFO:teuthology.orchestra.run.smithi073.stdout:1901 write 0xe0a4388 thru 0xe0b3c59 (0xf8d2 bytes) 2024-09-21T14:21:40.821 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.821 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.828 INFO:teuthology.orchestra.run.smithi073.stdout:1902 trunc from 0xe57d86a to 0x70dcd08 2024-09-21T14:21:40.830 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:40.838 INFO:teuthology.orchestra.run.smithi073.stdout:1903 read 0x2e28922 thru 0x2e2c2d2 (0x39b1 bytes) 2024-09-21T14:21:40.838 INFO:teuthology.orchestra.run.smithi073.stdout:1905 punch from 0x428cf60 to 0x429aad1, (0xdb71 bytes) 2024-09-21T14:21:40.838 INFO:teuthology.orchestra.run.smithi073.stdout:1906 punch from 0x411bbea to 0x41205fd, (0x4a13 bytes) 2024-09-21T14:21:40.838 INFO:teuthology.orchestra.run.smithi073.stdout:1907 write 0x3338c44 thru 0x33415a7 (0x8964 bytes) 2024-09-21T14:21:40.840 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:40.840+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:40.841 INFO:teuthology.orchestra.run.smithi073.stdout:1908 write 0x6659aa7 thru 0x6665a4e (0xbfa8 bytes) 2024-09-21T14:21:40.845 INFO:teuthology.orchestra.run.smithi073.stdout:1909 write 0x5a92df1 thru 0x5a9a13a (0x734a bytes) 2024-09-21T14:21:40.853 INFO:teuthology.orchestra.run.smithi073.stdout:1911 writesame 0x765a715 thru 0x7660e85 data_size 0x5b(0x6771 bytes) 2024-09-21T14:21:40.863 INFO:teuthology.orchestra.run.smithi073.stdout:1912 writesame 0xbc03f5e thru 0xbc0c2a2 data_size 0xd(0x8345 bytes) 2024-09-21T14:21:40.863 INFO:teuthology.orchestra.run.smithi073.stdout:1913 write 0xb0cbd85 thru 0xb0d76a6 (0xb922 bytes) 2024-09-21T14:21:40.871 INFO:teuthology.orchestra.run.smithi073.stdout:1915 trunc from 0xbc0c2a3 to 0x85d66c6 2024-09-21T14:21:40.893 INFO:teuthology.orchestra.run.smithi073.stdout:1917 writesame 0xe6d1f7c thru 0xe6d5c6d data_size 0x1d(0x3cf2 bytes) 2024-09-21T14:21:40.893 INFO:teuthology.orchestra.run.smithi073.stdout:1919 writesame 0x6cb1af7 thru 0x6cbd5d4 data_size 0x11(0xbade bytes) 2024-09-21T14:21:40.896 INFO:teuthology.orchestra.run.smithi073.stdout:1920 write 0x146536b thru 0x146fab5 (0xa74b bytes) 2024-09-21T14:21:40.898 INFO:teuthology.orchestra.run.smithi073.stdout:1921 read 0x403c32a thru 0x4047460 (0xb137 bytes) 2024-09-21T14:21:40.898 INFO:teuthology.orchestra.run.smithi073.stdout:1923 write 0x8da7d8e thru 0x8db1822 (0x9a95 bytes) 2024-09-21T14:21:40.901 INFO:teuthology.orchestra.run.smithi073.stdout:1924 read 0x264dc1 thru 0x2709e7 (0xbc27 bytes) 2024-09-21T14:21:40.903 INFO:teuthology.orchestra.run.smithi073.stdout:1925 writesame 0x93da1e7 thru 0x93e9e6d data_size 0x21(0xfc87 bytes) 2024-09-21T14:21:40.905 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.906 INFO:teuthology.orchestra.run.smithi073.stdout:1927 trunc from 0xe6d5c6e to 0xea2a344 2024-09-21T14:21:40.923 INFO:teuthology.orchestra.run.smithi073.stdout:1928 trunc from 0xea2a344 to 0x3bc1d7f 2024-09-21T14:21:40.933 INFO:teuthology.orchestra.run.smithi073.stdout:1929 trunc from 0x3bc1d7f to 0x328a8ec 2024-09-21T14:21:40.940 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:40.940 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:40.941 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:40.941+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:40.945 INFO:teuthology.orchestra.run.smithi073.stdout:1930 trunc from 0x328a8ec to 0x8a728d2 2024-09-21T14:21:40.949 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:40.950 INFO:teuthology.orchestra.run.smithi073.stdout:1931 read 0x2d0f165 thru 0x2d1b12e (0xbfca bytes) 2024-09-21T14:21:40.951 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.951 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.951 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.961 INFO:teuthology.orchestra.run.smithi073.stdout:1936 write 0xcf0b2b4 thru 0xcf10966 (0x56b3 bytes) 2024-09-21T14:21:40.964 INFO:teuthology.orchestra.run.smithi073.stdout:1937 punch from 0x640bbd3 to 0x64184d5, (0xc902 bytes) 2024-09-21T14:21:40.965 INFO:teuthology.orchestra.run.smithi073.stdout:1938 punch from 0x6463dda to 0x6472a4e, (0xec74 bytes) 2024-09-21T14:21:40.966 INFO:teuthology.orchestra.run.smithi073.stdout:1939 trunc from 0xcf10967 to 0xde2bb16 2024-09-21T14:21:40.971 INFO:teuthology.orchestra.run.smithi073.stdout:1941 read 0x63f0b99 thru 0x63f57a7 (0x4c0f bytes) 2024-09-21T14:21:40.971 INFO:teuthology.orchestra.run.smithi073.stdout:1947 punch from 0xa0806f9 to 0xa08d67d, (0xcf84 bytes) 2024-09-21T14:21:40.971 INFO:teuthology.orchestra.run.smithi073.stdout:1948 read 0xb7f9ebf thru 0xb80280c (0x894e bytes) 2024-09-21T14:21:40.971 INFO:teuthology.orchestra.run.smithi073.stdout:1949 punch from 0xac5e477 to 0xac6821d, (0x9da6 bytes) 2024-09-21T14:21:40.971 INFO:teuthology.orchestra.run.smithi073.stdout:1950 write 0x618877 thru 0x61f9fd (0x7187 bytes) 2024-09-21T14:21:40.972 INFO:teuthology.orchestra.run.smithi073.stdout:1952 read 0xcf8072a thru 0xcf85473 (0x4d4a bytes) 2024-09-21T14:21:40.972 INFO:teuthology.orchestra.run.smithi073.stdout:1953 read 0xe18d44 thru 0xe1bfb6 (0x3273 bytes) 2024-09-21T14:21:40.973 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.973 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.973 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.973 INFO:teuthology.orchestra.run.smithi073.stdout:1957 write 0xa0dab4 thru 0xa0fccf (0x221c bytes) 2024-09-21T14:21:40.977 INFO:teuthology.orchestra.run.smithi073.stdout:1958 writesame 0xb03f64f thru 0xb044a7d data_size 0x1(0x542f bytes) 2024-09-21T14:21:40.991 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:40.991 INFO:teuthology.orchestra.run.smithi073.stdout:1960 writesame 0x9707258 thru 0x9712570 data_size 0x3(0xb319 bytes) 2024-09-21T14:21:40.998 INFO:teuthology.orchestra.run.smithi073.stdout:1962 writesame 0x7b02cd4 thru 0x7b05865 data_size 0xb(0x2b92 bytes) 2024-09-21T14:21:41.002 INFO:teuthology.orchestra.run.smithi073.stdout:1964 write 0x237ca20 thru 0x23835d0 (0x6bb1 bytes) 2024-09-21T14:21:41.019 INFO:teuthology.orchestra.run.smithi073.stdout:1965 trunc from 0xde2bb16 to 0x239201c 2024-09-21T14:21:41.037 INFO:teuthology.orchestra.run.smithi073.stdout:1966 trunc from 0x239201c to 0x43dbcba 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:1970 punch from 0x35a6771 to 0x35b63fc, (0xfc8b bytes) 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:1973 punch from 0x3e8a454 to 0x3e912f0, (0x6e9c bytes) 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:1975 read 0x2dcff9d thru 0x2dd96c4 (0x9728 bytes) 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.042 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.042 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:41.042+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:41.054 INFO:teuthology.orchestra.run.smithi073.stdout:1981 write 0xb48a617 thru 0xb492eb3 (0x889d bytes) 2024-09-21T14:21:41.060 INFO:teuthology.orchestra.run.smithi073.stdout:1982 punch from 0x6ba26f1 to 0x6bb1304, (0xec13 bytes) 2024-09-21T14:21:41.061 INFO:teuthology.orchestra.run.smithi073.stdout:1984 punch from 0x5bce3c0 to 0x5bce867, (0x4a7 bytes) 2024-09-21T14:21:41.061 INFO:teuthology.orchestra.run.smithi073.stdout:1985 write 0x2580aa5 thru 0x2587ab9 (0x7015 bytes) 2024-09-21T14:21:41.063 INFO:teuthology.orchestra.run.smithi073.stdout:1986 clone 13 order 20 su 32768 sc 15 2024-09-21T14:21:41.096 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.096 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:41.146 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:41.146+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:41.243 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:41.244+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:41.247 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.247 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.255 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:41.268 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T14:21:41.268 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:21:41.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T14:21:41.337 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone12 from 0xb492eb4 (overlap 0x139867d) to 0x3eb47f 2024-09-21T14:21:41.344 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:41.344+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:41.410 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.410 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:41.436 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T14:21:41.444 INFO:teuthology.orchestra.run.smithi073.stdout:1988 punch from 0x6d33d45 to 0x6d35c60, (0x1f1b bytes) 2024-09-21T14:21:41.448 INFO:teuthology.orchestra.run.smithi073.stdout:1989 write 0x257dd8f thru 0x25851d8 (0x744a bytes) 2024-09-21T14:21:41.459 INFO:teuthology.orchestra.run.smithi073.stdout:1992 trunc from 0xb492eb4 to 0x5cd31d5 2024-09-21T14:21:41.460 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:41.460+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:41.478 INFO:teuthology.orchestra.run.smithi073.stdout:1993 punch from 0x2096390 to 0x20a1f4b, (0xbbbb bytes) 2024-09-21T14:21:41.478 INFO:teuthology.orchestra.run.smithi073.stdout:1994 read 0x2d13b13 thru 0x2d149ec (0xeda bytes) 2024-09-21T14:21:41.478 INFO:teuthology.orchestra.run.smithi073.stdout:1995 read 0x483d3e4 thru 0x48400da (0x2cf7 bytes) 2024-09-21T14:21:41.486 INFO:teuthology.orchestra.run.smithi073.stdout:1997 trunc from 0x5cd31d5 to 0xc46c42f 2024-09-21T14:21:41.491 INFO:teuthology.orchestra.run.smithi073.stdout:1999 write 0x7bfec0f thru 0x7c09943 (0xad35 bytes) 2024-09-21T14:21:41.495 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.495 INFO:teuthology.orchestra.run.smithi073.stdout:2004 read 0x355865f thru 0x3566a7c (0xe41e bytes) 2024-09-21T14:21:41.495 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.501 INFO:teuthology.orchestra.run.smithi073.stdout:2007 writesame 0xc5fbd5d thru 0xc601e15 data_size 0x1(0x60b9 bytes) 2024-09-21T14:21:41.501 INFO:teuthology.orchestra.run.smithi073.stdout:2008 writesame 0x97eb383 thru 0x97eb401 data_size 0x1(0x7f bytes) 2024-09-21T14:21:41.510 INFO:teuthology.orchestra.run.smithi073.stdout:2009 writesame 0xd7ceae6 thru 0xd7d5dc6 data_size 0x1(0x72e1 bytes) 2024-09-21T14:21:41.510 INFO:teuthology.orchestra.run.smithi073.stdout:2010 read 0x7b19ff2 thru 0x7b24306 (0xa315 bytes) 2024-09-21T14:21:41.510 INFO:teuthology.orchestra.run.smithi073.stdout:2012 write 0x52ede32 thru 0x52ef55a (0x1729 bytes) 2024-09-21T14:21:41.513 INFO:teuthology.orchestra.run.smithi073.stdout:2013 writesame 0x71de3a6 thru 0x71e289c data_size 0x21(0x44f7 bytes) 2024-09-21T14:21:41.517 INFO:teuthology.orchestra.run.smithi073.stdout:2014 writesame 0x1c41d24 thru 0x1c49de2 data_size 0x1(0x80bf bytes) 2024-09-21T14:21:41.537 INFO:teuthology.orchestra.run.smithi073.stdout:2015 punch from 0x5907a07 to 0x5916a88, (0xf081 bytes) 2024-09-21T14:21:41.537 INFO:teuthology.orchestra.run.smithi073.stdout:2016 read 0x7c4147 thru 0x7d2acc (0xe986 bytes) 2024-09-21T14:21:41.537 INFO:teuthology.orchestra.run.smithi073.stdout:2019 write 0x45a0e97 thru 0x45a2b99 (0x1d03 bytes) 2024-09-21T14:21:41.540 INFO:teuthology.orchestra.run.smithi073.stdout:2020 read 0xc8693fc thru 0xc86ceb2 (0x3ab7 bytes) 2024-09-21T14:21:41.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:41.546+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:41.546 INFO:teuthology.orchestra.run.smithi073.stdout:2022 trunc from 0xd7d5dc7 to 0x9369c47 2024-09-21T14:21:41.555 INFO:teuthology.orchestra.run.smithi073.stdout:2023 writesame 0x91b63c6 thru 0x91b9488 data_size 0x39(0x30c3 bytes) 2024-09-21T14:21:41.559 INFO:teuthology.orchestra.run.smithi073.stdout:2024 read 0xfbc332 thru 0xfc9068 (0xcd37 bytes) 2024-09-21T14:21:41.566 INFO:teuthology.orchestra.run.smithi073.stdout:2025 trunc from 0x9369c47 to 0x3c487da 2024-09-21T14:21:41.574 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.574 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.585 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:41.603 INFO:teuthology.orchestra.run.smithi073.stdout:2026 write 0x65c3e97 thru 0x65c433b (0x4a5 bytes) 2024-09-21T14:21:41.608 INFO:teuthology.orchestra.run.smithi073.stdout:2028 trunc from 0x65c433c to 0x41bd848 2024-09-21T14:21:41.624 INFO:teuthology.orchestra.run.smithi073.stdout:2030 read 0x1e124d4 thru 0x1e1fae9 (0xd616 bytes) 2024-09-21T14:21:41.627 INFO:teuthology.orchestra.run.smithi073.stdout:2032 trunc from 0x41bd848 to 0x6734f59 2024-09-21T14:21:41.649 INFO:teuthology.orchestra.run.smithi073.stdout:2033 writesame 0xe7578ed thru 0xe761da1 data_size 0xf(0xa4b5 bytes) 2024-09-21T14:21:41.657 INFO:teuthology.orchestra.run.smithi073.stdout:2035 trunc from 0xe761da2 to 0x7f0fbad 2024-09-21T14:21:41.662 INFO:teuthology.orchestra.run.smithi073.stdout:2037 read 0x789f819 thru 0x78ae7df (0xefc7 bytes) 2024-09-21T14:21:41.662 INFO:teuthology.orchestra.run.smithi073.stdout:2039 write 0x453aa50 thru 0x4548788 (0xdd39 bytes) 2024-09-21T14:21:41.670 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:41.670+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:41.670 INFO:teuthology.orchestra.run.smithi073.stdout:2040 write 0x5fd6112 thru 0x5fd7947 (0x1836 bytes) 2024-09-21T14:21:41.672 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.672 INFO:teuthology.orchestra.run.smithi073.stdout:2044 read 0x3ea354c thru 0x3eb21ca (0xec7f bytes) 2024-09-21T14:21:41.673 INFO:teuthology.orchestra.run.smithi073.stdout:2045 writesame 0x5897b3b thru 0x589a94a data_size 0x2c(0x2e10 bytes) 2024-09-21T14:21:41.676 INFO:teuthology.orchestra.run.smithi073.stdout:2046 read 0x51dfac7 thru 0x51e0b0c (0x1046 bytes) 2024-09-21T14:21:41.676 INFO:teuthology.orchestra.run.smithi073.stdout:2047 write 0x1ee3e28 thru 0x1ee594b (0x1b24 bytes) 2024-09-21T14:21:41.679 INFO:teuthology.orchestra.run.smithi073.stdout:2048 punch from 0x24b85ab to 0x24c330b, (0xad60 bytes) 2024-09-21T14:21:41.679 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.679 INFO:teuthology.orchestra.run.smithi073.stdout:2052 punch from 0x19f1c05 to 0x19f82dd, (0x66d8 bytes) 2024-09-21T14:21:41.679 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.689 INFO:teuthology.orchestra.run.smithi073.stdout:2054 write 0xafd320f thru 0xafd8041 (0x4e33 bytes) 2024-09-21T14:21:41.696 INFO:teuthology.orchestra.run.smithi073.stdout:2055 write 0x1ea2bd6 thru 0x1ea5764 (0x2b8f bytes) 2024-09-21T14:21:41.700 INFO:teuthology.orchestra.run.smithi073.stdout:2056 read 0x39b23bc thru 0x39b8e80 (0x6ac5 bytes) 2024-09-21T14:21:41.700 INFO:teuthology.orchestra.run.smithi073.stdout:2059 read 0x646e811 thru 0x646e926 (0x116 bytes) 2024-09-21T14:21:41.700 INFO:teuthology.orchestra.run.smithi073.stdout:2060 write 0x8f9fcda thru 0x8fa72c4 (0x75eb bytes) 2024-09-21T14:21:41.717 INFO:teuthology.orchestra.run.smithi073.stdout:2062 write 0xe977458 thru 0xe977714 (0x2bd bytes) 2024-09-21T14:21:41.720 INFO:teuthology.orchestra.run.smithi073.stdout:2063 write 0xa8ec6cb thru 0xa8ef7d8 (0x310e bytes) 2024-09-21T14:21:41.724 INFO:teuthology.orchestra.run.smithi073.stdout:2065 writesame 0x4fd02ab thru 0x4fd5748 data_size 0x1(0x549e bytes) 2024-09-21T14:21:41.735 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.736 INFO:teuthology.orchestra.run.smithi073.stdout:2068 read 0x681c3 thru 0x76b58 (0xe996 bytes) 2024-09-21T14:21:41.736 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.736 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.737 INFO:teuthology.orchestra.run.smithi073.stdout:2070 read 0x1bac39a thru 0x1bb351d (0x7184 bytes) 2024-09-21T14:21:41.737 INFO:teuthology.orchestra.run.smithi073.stdout:2071 punch from 0xcf08fe6 to 0xcf0acfb, (0x1d15 bytes) 2024-09-21T14:21:41.737 INFO:teuthology.orchestra.run.smithi073.stdout:2072 writesame 0xbcf2d9 thru 0xbdc416 data_size 0x1(0xd13e bytes) 2024-09-21T14:21:41.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:41.747 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:41.746+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:41.762 INFO:teuthology.orchestra.run.smithi073.stdout:2073 read 0x6f1b8dc thru 0x6f27fe1 (0xc706 bytes) 2024-09-21T14:21:41.762 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.762 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.762 INFO:teuthology.orchestra.run.smithi073.stdout:2079 punch from 0xcd36047 to 0xcd386cb, (0x2684 bytes) 2024-09-21T14:21:41.762 INFO:teuthology.orchestra.run.smithi073.stdout:2080 writesame 0x102893e thru 0x102c6d0 data_size 0x1(0x3d93 bytes) 2024-09-21T14:21:41.773 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:41.775 INFO:teuthology.orchestra.run.smithi073.stdout:2081 read 0xa40ff1a thru 0xa41b369 (0xb450 bytes) 2024-09-21T14:21:41.775 INFO:teuthology.orchestra.run.smithi073.stdout:2083 writesame 0xf96cda thru 0xf97283 data_size 0xa(0x5aa bytes) 2024-09-21T14:21:41.790 INFO:teuthology.orchestra.run.smithi073.stdout:2084 trunc from 0xe977715 to 0x6324326 2024-09-21T14:21:41.809 INFO:teuthology.orchestra.run.smithi073.stdout:2085 trunc from 0x6324326 to 0xf6e1f 2024-09-21T14:21:41.847 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:41.847+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:41.870 INFO:teuthology.orchestra.run.smithi073.stdout:2086 write 0xe18b53b thru 0xe18dd04 (0x27ca bytes) 2024-09-21T14:21:41.873 INFO:teuthology.orchestra.run.smithi073.stdout:2087 writesame 0x3be8fe7 thru 0x3bed1c0 data_size 0x1(0x41da bytes) 2024-09-21T14:21:41.883 INFO:teuthology.orchestra.run.smithi073.stdout:2088 write 0x79efef1 thru 0x79fa654 (0xa764 bytes) 2024-09-21T14:21:41.888 INFO:teuthology.orchestra.run.smithi073.stdout:2089 write 0x7a16327 thru 0x7a234c7 (0xd1a1 bytes) 2024-09-21T14:21:41.903 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:41.903 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:41.905 INFO:teuthology.orchestra.run.smithi073.stdout:2094 trunc from 0xe18dd05 to 0x2ea4b74 2024-09-21T14:21:41.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:41.922 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:41.922 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:41.925 INFO:teuthology.orchestra.run.smithi073.stdout:2095 write 0x68c7660 thru 0x68c89b4 (0x1355 bytes) 2024-09-21T14:21:41.928 INFO:teuthology.orchestra.run.smithi073.stdout:2096 write 0x513879b thru 0x513d834 (0x509a bytes) 2024-09-21T14:21:41.931 INFO:teuthology.orchestra.run.smithi073.stdout:2098 read 0x56aec95 thru 0x56bd029 (0xe395 bytes) 2024-09-21T14:21:41.940 INFO:teuthology.orchestra.run.smithi073.stdout:2099 write 0x9979ad5 thru 0x99823f4 (0x8920 bytes) 2024-09-21T14:21:41.941 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:41.941 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:41.947 INFO:teuthology.orchestra.run.smithi073.stdout:2100 writesame 0x7a76357 thru 0x7a7e249 data_size 0x45(0x7ef3 bytes) 2024-09-21T14:21:41.954 INFO:teuthology.orchestra.run.smithi073.stdout:2101 read 0x3114cea thru 0x3122f80 (0xe297 bytes) 2024-09-21T14:21:41.955 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:41.955+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:41.955 INFO:teuthology.orchestra.run.smithi073.stdout:2103 writesame 0x5126212 thru 0x512ed10 data_size 0x1d(0x8aff bytes) 2024-09-21T14:21:41.958 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:41.958 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:2105 read 0x92e4c86 thru 0x92f0405 (0xb780 bytes) 2024-09-21T14:21:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:2106 read 0x886585 thru 0x8939a0 (0xd41c bytes) 2024-09-21T14:21:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:2108 read 0x82f5b21 thru 0x82faa3e (0x4f1e bytes) 2024-09-21T14:21:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:41.963 INFO:teuthology.orchestra.run.smithi073.stdout:2110 read 0x5852af3 thru 0x5858e87 (0x6395 bytes) 2024-09-21T14:21:41.963 INFO:teuthology.orchestra.run.smithi073.stdout:2111 punch from 0x7b4a6de to 0x7b4d6ae, (0x2fd0 bytes) 2024-09-21T14:21:41.974 INFO:teuthology.orchestra.run.smithi073.stdout:2112 trunc from 0x99823f5 to 0xef77ad 2024-09-21T14:21:41.984 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:41.984 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:41.985 INFO:teuthology.orchestra.run.smithi073.stdout:2113 read 0xa1ed59 thru 0xa2ddae (0xf056 bytes) 2024-09-21T14:21:41.995 INFO:teuthology.orchestra.run.smithi073.stdout:2114 write 0x499911e thru 0x49a165c (0x853f bytes) 2024-09-21T14:21:41.999 INFO:teuthology.orchestra.run.smithi073.stdout:2118 read 0x3620c94 thru 0x3627586 (0x68f3 bytes) 2024-09-21T14:21:42.002 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:42.002 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:42.005 INFO:teuthology.orchestra.run.smithi073.stdout:2119 write 0x594daa5 thru 0x595b9ae (0xdf0a bytes) 2024-09-21T14:21:42.017 INFO:teuthology.orchestra.run.smithi073.stdout:2120 writesame 0x74f8b1d thru 0x750892c data_size 0x1e(0xfe10 bytes) 2024-09-21T14:21:42.017 INFO:teuthology.orchestra.run.smithi073.stdout:2121 clone 14 order 24 su 65536 sc 11 2024-09-21T14:21:42.020 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:42.020 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:42.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:42.036 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:42.048 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:42.048+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:42.053 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:42.053 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:42.060 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:42.060 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:42.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:42.149 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:42.149+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:42.171 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:42.172 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:42.172 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:42.172 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:42.172 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:42.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:42.251 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:42.251+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:42.284 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T14:21:42.285 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:42.285 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:42.285 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2576 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18350080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 18.046099774000002, 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018251078, 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2576 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238398+0000", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238397+0000", 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238400+0000", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.74e-06 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238406+0000", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2369999999999997e-06 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238411+0000", 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.639e-06 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256452+0000", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018040901000000002 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256536+0000", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4635999999999994e-05 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256649+0000", 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00011244299999999999 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.289 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 18.046082427000002, 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.290 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:21:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:21:42.293 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.688101063, 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:21:42.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.688083736999999, 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.299 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:21:42.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.688066223, 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.303 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:42.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:21:42.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:42.307 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.688048969, 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:21:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:21:42.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.688021119, 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.312 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.313 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:21:42.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:21:42.315 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687999515, 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.316 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:21:42.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687982517, 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.320 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:21:42.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:21:42.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687965497, 2024-09-21T14:21:42.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:21:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:21:42.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687948047000001, 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.329 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.330 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687930909, 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.333 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.337 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687907682000001, 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:42.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.340 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.341 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687891408, 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.342 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687867036, 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687844469, 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.350 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:21:42.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687827548, 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:21:42.354 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:21:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:21:42.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687811049, 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.358 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:21:42.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:21:42.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687790052, 2024-09-21T14:21:42.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:21:42.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:21:42.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.366 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 12.687767994, 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:42.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:21:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.370 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.371 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:42.371 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:42.371 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:42.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:21:42.373 INFO:teuthology.orchestra.run.smithi073.stdout:flattening image image_client.0-clone13 2024-09-21T14:21:42.373 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:42.372+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:42.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:42.466 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T14:21:42.472 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:42.473+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:42.490 INFO:teuthology.orchestra.run.smithi073.stdout:2123 write 0xe842868 thru 0xe84a69c (0x7e35 bytes) 2024-09-21T14:21:42.494 INFO:teuthology.orchestra.run.smithi073.stdout:2124 read 0x271bf8c thru 0x2726da2 (0xae17 bytes) 2024-09-21T14:21:42.494 INFO:teuthology.orchestra.run.smithi073.stdout:2126 punch from 0xb9c888b to 0xb9d7219, (0xe98e bytes) 2024-09-21T14:21:42.494 INFO:teuthology.orchestra.run.smithi073.stdout:2127 read 0xa3fd2e0 thru 0xa3ff70b (0x242c bytes) 2024-09-21T14:21:42.494 INFO:teuthology.orchestra.run.smithi073.stdout:2128 read 0xd6411e7 thru 0xd64f6cc (0xe4e6 bytes) 2024-09-21T14:21:42.494 INFO:teuthology.orchestra.run.smithi073.stdout:2131 write 0x4234ffd thru 0x42433f3 (0xe3f7 bytes) 2024-09-21T14:21:42.505 INFO:teuthology.orchestra.run.smithi073.stdout:2132 write 0xe18928b thru 0xe1932e7 (0xa05d bytes) 2024-09-21T14:21:42.510 INFO:teuthology.orchestra.run.smithi073.stdout:2133 read 0xcfcb5f2 thru 0xcfd45c4 (0x8fd3 bytes) 2024-09-21T14:21:42.510 INFO:teuthology.orchestra.run.smithi073.stdout:2136 writesame 0x2f15659 thru 0x2f1cd20 data_size 0x54(0x76c8 bytes) 2024-09-21T14:21:42.517 INFO:teuthology.orchestra.run.smithi073.stdout:2137 write 0x987356c thru 0x98786e1 (0x5176 bytes) 2024-09-21T14:21:42.523 INFO:teuthology.orchestra.run.smithi073.stdout:2138 write 0x3bef096 thru 0x3bf4e11 (0x5d7c bytes) 2024-09-21T14:21:42.534 INFO:teuthology.orchestra.run.smithi073.stdout:2139 writesame 0x3af2b9 thru 0x3beff7 data_size 0x1(0xfd3f bytes) 2024-09-21T14:21:42.538 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:42.538 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:42.538 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:42.538 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:42.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:42.562 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:42.562 INFO:teuthology.orchestra.run.smithi073.stdout:2142 punch from 0x675d31d to 0x6765d6e, (0x8a51 bytes) 2024-09-21T14:21:42.562 INFO:teuthology.orchestra.run.smithi073.stdout:2143 write 0x3659145 thru 0x365e61a (0x54d6 bytes) 2024-09-21T14:21:42.569 INFO:teuthology.orchestra.run.smithi073.stdout:2144 write 0x4548fbf thru 0x4558628 (0xf66a bytes) 2024-09-21T14:21:42.573 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:42.573+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:42.575 INFO:teuthology.orchestra.run.smithi073.stdout:2145 clone 15 order 23 su 4194304 sc 2 2024-09-21T14:21:42.668 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:42.668 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:42.668 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:42.668 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:42.668 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:42.673 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:42.674+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:42.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:42.774 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:42.774+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:42.819 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11320 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508115945, 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041544266000000003, 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11320 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311097+0000", 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7999999999999993e-07 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311102+0000", 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1620000000000001e-06 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311110+0000", 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0819999999999999e-06 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352476+0000", 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041366078000000001 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352554+0000", 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7965000000000003e-05 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352641+0000", 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7194999999999994e-05 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11321 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.824 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508100706, 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041652646000000002, 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11321 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311112+0000", 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311116+0000", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311119+0000", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.45e-06 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311122+0000", 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3690000000000001e-06 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352670+0000", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041548359 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352714+0000", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3461000000000003e-05 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352765+0000", 2024-09-21T14:21:42.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0911000000000001e-05 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11323 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 22528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508088807, 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041853842000000002, 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11323 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311124+0000", 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311125+0000", 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.051e-06 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311128+0000", 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.886e-06 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311130+0000", 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.34e-06 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352794+0000", 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041663797000000002 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352871+0000", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7476000000000002e-05 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.352978+0000", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106492 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11324 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:42.833 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508080704999999, 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042007186000000002, 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11324 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311132+0000", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311133+0000", 2024-09-21T14:21:42.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559999999999999e-06 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311136+0000", 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7829999999999999e-06 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311138+0000", 2024-09-21T14:21:42.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.362e-06 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353020+0000", 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041881717999999998 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353078+0000", 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8344999999999998e-05 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353139+0000", 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1124000000000006e-05 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11325 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 59392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508072769, 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042198257000000003, 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.838 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11325 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311140+0000", 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311141+0000", 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.017e-06 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311143+0000", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7719999999999999e-06 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311146+0000", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.362e-06 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353178+0000", 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042032067999999999 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353257+0000", 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9500999999999997e-05 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353338+0000", 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0745e-05 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.842 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11326 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508063902, 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042324452999999998, 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11326 2024-09-21T14:21:42.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311149+0000", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311150+0000", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0210000000000001e-06 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311152+0000", 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7750000000000001e-06 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311155+0000", 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3719999999999999e-06 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353368+0000", 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042213186999999999 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353419+0000", 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1542000000000002e-05 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353473+0000", 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3761000000000002e-05 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11328 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.508056025, 2024-09-21T14:21:42.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042440522000000001, 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11328 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311156+0000", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311157+0000", 2024-09-21T14:21:42.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8599999999999996e-07 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311160+0000", 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8229999999999999e-06 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311163+0000", 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353496+0000", 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042333000000000003 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353546+0000", 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0818e-05 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353597+0000", 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0796e-05 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11330 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 92160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.507931383000001, 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042450188999999999, 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11330 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311281+0000", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311164+0000", 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9998832 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311284+0000", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119777 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311294+0000", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0342e-05 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.311300+0000", 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1580000000000004e-06 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353629+0000", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.042329929000000002 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353678+0000", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.829e-05 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.353731+0000", 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3687999999999998e-05 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11341 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 182272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.506069241000001, 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041454537999999999, 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11341 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313143+0000", 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313145+0000", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2730000000000001e-06 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313151+0000", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313156+0000", 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.724e-06 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354503+0000", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041346489 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354549+0000", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5985000000000001e-05 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354598+0000", 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9375000000000001e-05 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.859 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11342 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.506052111000001, 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041568147999999999, 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11342 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.860 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313161+0000", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313160+0000", 2024-09-21T14:21:42.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313162+0000", 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313169+0000", 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.528e-06 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313173+0000", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3959999999999999e-06 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354626+0000", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041452799999999998 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354676+0000", 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9543999999999999e-05 2024-09-21T14:21:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354729+0000", 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3103000000000003e-05 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11344 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.506035299000001, 2024-09-21T14:21:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041669309000000002, 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11344 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313177+0000", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313179+0000", 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.075e-06 2024-09-21T14:21:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313185+0000", 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7769999999999999e-06 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313190+0000", 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8520000000000003e-06 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354750+0000", 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041560198999999999 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354798+0000", 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7976000000000003e-05 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354847+0000", 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8797000000000001e-05 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11345 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.506018106999999, 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041777117000000003, 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11345 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313195+0000", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313194+0000", 2024-09-21T14:21:42.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313196+0000", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1780000000000002e-06 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313202+0000", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1190000000000002e-06 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313207+0000", 2024-09-21T14:21:42.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4390000000000002e-06 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354874+0000", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041666789000000003 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354922+0000", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8096000000000002e-05 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.354972+0000", 2024-09-21T14:21:42.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9901999999999998e-05 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11347 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.506001804, 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041891943000000001, 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.873 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11347 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313211+0000", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313213+0000", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0389999999999999e-06 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313221+0000", 2024-09-21T14:21:42.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8149999999999995e-06 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313231+0000", 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2199999999999998e-06 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355002+0000", 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041771097 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355051+0000", 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9821000000000003e-05 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355103+0000", 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1292999999999998e-05 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11348 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 276480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:42.877 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.505954496999999, 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041973968, 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11348 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313258+0000", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313261+0000", 2024-09-21T14:21:42.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9119999999999998e-06 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313268+0000", 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8520000000000004e-06 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.313273+0000", 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8960000000000002e-06 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355130+0000", 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041856972999999999 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355180+0000", 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9264999999999997e-05 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.355232+0000", 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2617999999999997e-05 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11366 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 436224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.503994548, 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041505316, 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11366 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315218+0000", 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315220+0000", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.024e-06 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315226+0000", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8640000000000001e-06 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.315231+0000", 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.882e-06 2024-09-21T14:21:42.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356622+0000", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041391773999999999 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356674+0000", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1255999999999999e-05 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.356723+0000", 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9867000000000002e-05 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.885 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11380 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.502108173, 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041491635999999998, 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.886 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11380 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317104+0000", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317105+0000", 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.173e-06 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317108+0000", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8770000000000002e-06 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317111+0000", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4830000000000002e-06 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358429+0000", 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041318393000000002 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358514+0000", 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5115000000000001e-05 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358596+0000", 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1808e-05 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11381 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 129024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.502099778, 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041612616999999998, 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11381 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317113+0000", 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317114+0000", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.144e-06 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317117+0000", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8080000000000001e-06 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317119+0000", 2024-09-21T14:21:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358624+0000", 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041504486 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358670+0000", 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6217999999999997e-05 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358725+0000", 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5733999999999999e-05 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11383 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.502091420999999, 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041761832999999998, 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11383 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317121+0000", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317123+0000", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300000000000001e-06 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317127+0000", 2024-09-21T14:21:42.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8300000000000003e-06 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317130+0000", 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5229999999999998e-06 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358759+0000", 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041629475999999999 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358831+0000", 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1283000000000002e-05 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358883+0000", 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2349999999999999e-05 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11385 3.f 3:f6f3b713:::rbd_data.1121cf46bdbd.0000000000000010:7 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:42.899 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.502058769, 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041857556999999997, 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11385 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:42.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317154+0000", 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317153+0000", 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317157+0000", 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.614e-06 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317164+0000", 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9720000000000003e-06 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317168+0000", 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4870000000000004e-06 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358912+0000", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041743495999999998 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.358952+0000", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0462e-05 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359011+0000", 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9110000000000002e-05 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.903 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:11386 3.f 3:f04d9f58:::rbd_data.1121cf46bdbd.000000000000000f:7 [read 186368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.502040887, 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041945596000000002, 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 11386 2024-09-21T14:21:42.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317172+0000", 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:42.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317171+0000", 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317173+0000", 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4610000000000001e-06 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317176+0000", 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.337e-06 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.317179+0000", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.605e-06 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359023+0000", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041844488999999999 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359069+0000", 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5667000000000002e-05 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:42.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:30.359117+0000", 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8371000000000002e-05 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:42.908 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:42.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:42.910 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:42.909+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:43.010 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:43.011+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:43.011 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.011 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:43.011 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:43.011 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:43.111 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:43.111+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:43.121 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.121 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:43.121 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:43.121 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:43.121 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.129 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:43.214 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:43.214+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6909 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678489+0000", 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592910183000001, 2024-09-21T14:21:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015920490999999998, 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6909 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678489+0000", 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678489+0000", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678488+0000", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678491+0000", 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3099999999999999e-06 2024-09-21T14:21:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678497+0000", 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8499999999999999e-06 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678501+0000", 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8350000000000003e-06 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694166+0000", 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015664315000000002 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694302+0000", 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013619700000000001 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694409+0000", 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010744800000000001 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6910 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 339968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678504+0000", 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592894942000001, 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016125146999999999, 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6910 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678504+0000", 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678504+0000", 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678504+0000", 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678506+0000", 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9309999999999998e-06 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678511+0000", 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5180000000000004e-06 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678515+0000", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0810000000000004e-06 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694438+0000", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015922397000000001 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694539+0000", 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010156 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694629+0000", 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9882000000000004e-05 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6911 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678519+0000", 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592879526000001, 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016317213000000001, 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6911 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678519+0000", 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678519+0000", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678519+0000", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678526+0000", 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.601e-06 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678532+0000", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2580000000000001e-06 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678537+0000", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.288e-06 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694662+0000", 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016124919000000001 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694752+0000", 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0025000000000006e-05 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694837+0000", 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4458999999999999e-05 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6912 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678542+0000", 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592857433000001, 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016482453000000001, 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6912 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678542+0000", 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678542+0000", 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678541+0000", 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678544+0000", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678549+0000", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6960000000000002e-06 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678553+0000", 2024-09-21T14:21:43.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2230000000000001e-06 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694872+0000", 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016318322999999999 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.694951+0000", 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8943000000000006e-05 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695024+0000", 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3302999999999999e-05 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6913 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678557+0000", 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592841504999999, 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016624137000000001, 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.290 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6913 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678557+0000", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678557+0000", 2024-09-21T14:21:43.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678557+0000", 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678559+0000", 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678565+0000", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7710000000000003e-06 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678569+0000", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2130000000000002e-06 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695064+0000", 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016494754 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695122+0000", 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7849000000000001e-05 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695182+0000", 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9888e-05 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.294 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6914 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678573+0000", 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592825642999999, 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016774188999999998, 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6914 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678573+0000", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678573+0000", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678573+0000", 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678575+0000", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678581+0000", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7980000000000002e-06 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678585+0000", 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2230000000000001e-06 2024-09-21T14:21:43.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695209+0000", 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016623921 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695296+0000", 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7167000000000001e-05 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695348+0000", 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1053000000000001e-05 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6915 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678590+0000", 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592809337, 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016910789999999998, 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6915 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678590+0000", 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678590+0000", 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678589+0000", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678591+0000", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.091e-06 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678597+0000", 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678602+0000", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4630000000000003e-06 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695386+0000", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01678373 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695432+0000", 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6635000000000003e-05 2024-09-21T14:21:43.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695500+0000", 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8311000000000004e-05 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:6916 3.1a 3:5b2f2c6e:::rbd_data.1121cf46bdbd.0000000000000027:7 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:28.678606+0000", 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.592792936, 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017052557999999999, 2024-09-21T14:21:43.303 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 6916 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678606+0000", 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678606+0000", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678606+0000", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678609+0000", 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678615+0000", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1020000000000002e-06 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.678620+0000", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8080000000000004e-06 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695526+0000", 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016906261999999998 2024-09-21T14:21:43.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695595+0000", 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8998000000000006e-05 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:28.695659+0000", 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3330999999999994e-05 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.050462125999999, 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:21:43.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:21:43.312 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30892 3.1c 3:3f6fce54:::rbd_data.1121836cb251.0000000000000002:a [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480112+0000", 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7912868839999998, 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015832290999999998, 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30892 2024-09-21T14:21:43.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480112+0000", 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480112+0000", 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480112+0000", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480118+0000", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8680000000000004e-06 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480121+0000", 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1549999999999999e-06 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480123+0000", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2800000000000002e-06 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.495846+0000", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015722955 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.495893+0000", 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6508999999999998e-05 2024-09-21T14:21:43.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.495944+0000", 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1725999999999999e-05 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30893 3.1c 3:3f6fce54:::rbd_data.1121836cb251.0000000000000002:a [read 1343488~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480125+0000", 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7912735050000004, 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01594576, 2024-09-21T14:21:43.317 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30893 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480125+0000", 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480125+0000", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480125+0000", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480140+0000", 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5065999999999999e-05 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480149+0000", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3480000000000004e-06 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480153+0000", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7600000000000002e-06 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.495968+0000", 2024-09-21T14:21:43.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015814582000000001 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496018+0000", 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0337000000000003e-05 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496071+0000", 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2901000000000001e-05 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30894 3.1c 3:3f6fce54:::rbd_data.1121836cb251.0000000000000002:a [read 1409024~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480158+0000", 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7912407190000001, 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016035093, 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30894 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.322 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480158+0000", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480158+0000", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480158+0000", 2024-09-21T14:21:43.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480160+0000", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.249e-06 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480165+0000", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2510000000000002e-06 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480169+0000", 2024-09-21T14:21:43.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1579999999999998e-06 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496095+0000", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015925721 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496142+0000", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6499000000000003e-05 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496193+0000", 2024-09-21T14:21:43.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1804000000000001e-05 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30911 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1703936~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480174+0000", 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.791225474, 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016137479, 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30911 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480174+0000", 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480174+0000", 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480173+0000", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480175+0000", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2000000000000001e-06 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480181+0000", 2024-09-21T14:21:43.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3959999999999997e-06 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480186+0000", 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9119999999999997e-06 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496217+0000", 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016030987999999999 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496258+0000", 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1424999999999997e-05 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496311+0000", 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2899000000000003e-05 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30912 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1769472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480190+0000", 2024-09-21T14:21:43.330 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7912093550000003, 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016245071999999999, 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30912 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480190+0000", 2024-09-21T14:21:43.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480190+0000", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480189+0000", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480191+0000", 2024-09-21T14:21:43.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480197+0000", 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3279999999999996e-06 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480201+0000", 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496334+0000", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016133413999999999 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496382+0000", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7299999999999998e-05 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496435+0000", 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.295e-05 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.334 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30919 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480205+0000", 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7911939740000005, 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016346398000000002, 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30919 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.335 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480205+0000", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480205+0000", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480205+0000", 2024-09-21T14:21:43.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480207+0000", 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0389999999999999e-06 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480214+0000", 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9249999999999998e-06 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480218+0000", 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.566e-06 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496457+0000", 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016239000999999999 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496499+0000", 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1502999999999999e-05 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496551+0000", 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669e-05 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30920 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1802240~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480222+0000", 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.791176986, 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016451358999999999, 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.339 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30920 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480222+0000", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480222+0000", 2024-09-21T14:21:43.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480222+0000", 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480224+0000", 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0219999999999999e-06 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480229+0000", 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2310000000000004e-06 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480239+0000", 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0518e-05 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496574+0000", 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016334255999999998 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496621+0000", 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6916999999999998e-05 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496673+0000", 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2750999999999997e-05 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30921 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1835008~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.343 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480244+0000", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7911555000000003, 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016545799, 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30921 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480244+0000", 2024-09-21T14:21:43.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480244+0000", 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480243+0000", 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480245+0000", 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1890000000000001e-06 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480251+0000", 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7690000000000002e-06 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480255+0000", 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2540000000000003e-06 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496696+0000", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016440867000000001 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496738+0000", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2267e-05 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496789+0000", 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0852e-05 2024-09-21T14:21:43.347 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30922 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1900544~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480260+0000", 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7911388590000001, 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016650965, 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.348 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30922 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480260+0000", 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480260+0000", 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480260+0000", 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480262+0000", 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8059999999999999e-06 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480275+0000", 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3407999999999999e-05 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480279+0000", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4170000000000002e-06 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496812+0000", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016532739000000001 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496858+0000", 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5716000000000001e-05 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496911+0000", 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30929 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1867776~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480284+0000", 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7911146770000004, 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016751491, 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.352 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30929 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480284+0000", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480284+0000", 2024-09-21T14:21:43.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480284+0000", 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480286+0000", 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3120000000000001e-06 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480292+0000", 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8420000000000001e-06 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480296+0000", 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1819999999999999e-06 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496934+0000", 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016637814000000001 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.496976+0000", 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1514000000000003e-05 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.497036+0000", 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0192000000000002e-05 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.356 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:30930 3.1c 3:382e2da1:::rbd_data.1121836cb251.0000000000000006:a [read 1933312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:38.480300+0000", 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7910988100000003, 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016855670999999999, 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 30930 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480300+0000", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480300+0000", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480300+0000", 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480302+0000", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.928e-06 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480307+0000", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6609999999999997e-06 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.480312+0000", 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3000000000000003e-06 2024-09-21T14:21:43.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.497051+0000", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01673906 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.497100+0000", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9338000000000001e-05 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:38.497156+0000", 2024-09-21T14:21:43.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5699999999999999e-05 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.361 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.361 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:21:43.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:43.364 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:21:43.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:21:43.364 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone14 from 0xe84a69d (overlap 0x750892d) to 0x231c2c0 2024-09-21T14:21:43.365 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:43.362+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:43.432 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T14:21:43.473 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:43.473+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:43.523 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.524 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:43.524 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:43.524 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:43.564 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:43.564+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:43.572 INFO:teuthology.orchestra.run.smithi073.stdout:2146 write 0x3072575 thru 0x307d629 (0xb0b5 bytes) 2024-09-21T14:21:43.596 INFO:teuthology.orchestra.run.smithi073.stdout:2147 trunc from 0xe84a69d to 0x9fbaf46 2024-09-21T14:21:43.607 INFO:teuthology.orchestra.run.smithi073.stdout:2148 trunc from 0x9fbaf46 to 0x954847e 2024-09-21T14:21:43.630 INFO:teuthology.orchestra.run.smithi073.stdout:2149 trunc from 0x954847e to 0x2c878d 2024-09-21T14:21:43.654 INFO:teuthology.orchestra.run.smithi073.stdout:2150 read 0x1752d thru 0x244b3 (0xcf87 bytes) 2024-09-21T14:21:43.654 INFO:teuthology.orchestra.run.smithi073.stdout:2151 read 0x1a3ff8 thru 0x1b133d (0xd346 bytes) 2024-09-21T14:21:43.654 INFO:teuthology.orchestra.run.smithi073.stdout:2152 read 0x151744 thru 0x154ec8 (0x3785 bytes) 2024-09-21T14:21:43.655 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:43.662 INFO:teuthology.orchestra.run.smithi073.stdout:2155 write 0x12ddc13 thru 0x12e94ed (0xb8db bytes) 2024-09-21T14:21:43.664 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:43.664+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:43.667 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:43.667 INFO:teuthology.orchestra.run.smithi073.stdout:2157 punch from 0x7cb188 to 0x7cea08, (0x3880 bytes) 2024-09-21T14:21:43.667 INFO:teuthology.orchestra.run.smithi073.stdout:2158 clone 16 order 20 su 16384 sc 15 2024-09-21T14:21:43.699 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.699 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:43.699 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:43.699 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:43.699 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:43.765 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:43.765+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:43.852 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:43.852 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:43.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.459734331, 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.853 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:43.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:21:43.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:21:43.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.45972343, 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.857 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:43.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:21:43.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.459713022000001, 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:21:43.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.459705080999999, 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.866 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:43.870 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382967096, 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.874 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382948686000001, 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:21:43.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:21:43.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382922937, 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.879 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:21:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382901908000001, 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:21:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382876058000001, 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:21:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.892 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382857579, 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:21:43.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382839456999999, 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.901 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382819095, 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:43.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.905 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382799403, 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:21:43.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:43.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:21:43.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382782425, 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.910 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:21:43.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382762753, 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:21:43.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382733290000001, 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:21:43.919 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:21:43.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.923 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382691044, 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.924 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:43.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382672056000001, 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:21:43.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382648698000001, 2024-09-21T14:21:43.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:21:43.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:21:43.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.936 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.382625344999999, 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:43.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:43.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:21:43.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:43.940 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:43.941 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:43.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:43.942 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:43.942+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:44.042 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:44.043+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:44.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:44.086 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:44.086 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:44.087 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:44.087 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:44.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:44.143 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:44.143+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:44.238 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:44.238 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:44.238 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:44.238 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:44.238 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:44.244 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:44.244+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:44.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:44.299 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:21:44.299 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:21:44.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:21:44.356 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:44.356+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:44.369 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone15 from 0x12e94ee (overlap 0x2c878d) to 0x1643c6 2024-09-21T14:21:44.390 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:44.390 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:44.409 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:44.409 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:44.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:44.409 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:44.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 34.866822542000001, 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.410 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:44.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:44.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.416 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9347 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811817983999999, 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031239744999999999, 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.417 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9347 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596944+0000", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596945+0000", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7199999999999997e-07 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596947+0000", 2024-09-21T14:21:44.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7609999999999999e-06 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596950+0000", 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.604e-06 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628112+0000", 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031161525999999998 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628148+0000", 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6531e-05 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628184+0000", 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5559999999999998e-05 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9348 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.421 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.81180992, 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031324221999999999, 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9348 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596952+0000", 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596953+0000", 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6299999999999993e-07 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596956+0000", 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9950000000000001e-06 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596961+0000", 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.826e-06 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628204+0000", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031243602999999998 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628240+0000", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5958999999999999e-05 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628276+0000", 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.608e-05 2024-09-21T14:21:44.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9349 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811797785, 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031404135, 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9349 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596964+0000", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596966+0000", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2859999999999998e-06 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596972+0000", 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:21:44.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596977+0000", 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6090000000000003e-06 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628296+0000", 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031319461999999999 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628332+0000", 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6266000000000001e-05 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628368+0000", 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5994999999999997e-05 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9350 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:44.430 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811781435, 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031481451000000001, 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9350 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596980+0000", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596982+0000", 2024-09-21T14:21:44.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4300000000000001e-06 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596992+0000", 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0679999999999999e-05 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596995+0000", 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6070000000000002e-06 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628389+0000", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031393873000000003 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628426+0000", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7013e-05 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628462+0000", 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6117e-05 2024-09-21T14:21:44.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9351 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811764574, 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031558256999999999, 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9351 2024-09-21T14:21:44.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596997+0000", 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.596998+0000", 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3200000000000001e-06 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597001+0000", 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9129999999999999e-06 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597004+0000", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.435e-06 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628483+0000", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031478902000000003 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628520+0000", 2024-09-21T14:21:44.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7670999999999999e-05 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628556+0000", 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5247000000000003e-05 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9352 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811756065000001, 2024-09-21T14:21:44.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031641458999999997, 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9352 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597006+0000", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597012+0000", 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4790000000000001e-06 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597015+0000", 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.174e-06 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597019+0000", 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.461e-06 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628575+0000", 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031556661 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628611+0000", 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5345999999999997e-05 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628647+0000", 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6548999999999999e-05 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9353 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811735794000001, 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031734509000000001, 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9353 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597026+0000", 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597028+0000", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597034+0000", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9039999999999997e-06 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597038+0000", 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.639e-06 2024-09-21T14:21:44.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628676+0000", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031637327 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628725+0000", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9208e-05 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628761+0000", 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5608000000000002e-05 2024-09-21T14:21:44.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9354 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811719006000001, 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031811304999999998, 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9354 2024-09-21T14:21:44.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597043+0000", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597042+0000", 2024-09-21T14:21:44.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597045+0000", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3999999999999999e-06 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597050+0000", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4310000000000002e-06 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597054+0000", 2024-09-21T14:21:44.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.489e-06 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628780+0000", 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031726733999999999 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628817+0000", 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6525e-05 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628854+0000", 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7283000000000002e-05 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9355 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811704593, 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031891615999999998, 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9355 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597057+0000", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597059+0000", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.854e-06 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597064+0000", 2024-09-21T14:21:44.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0019999999999997e-06 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597069+0000", 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1429999999999997e-06 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628874+0000", 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031805319999999998 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628912+0000", 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8102000000000002e-05 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628949+0000", 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6514999999999999e-05 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9356 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.456 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811688779000001, 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031979298000000003, 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9356 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:44.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597073+0000", 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597075+0000", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1210000000000001e-06 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597082+0000", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.024e-06 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597086+0000", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.628970+0000", 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031883504 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629014+0000", 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4458e-05 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629052+0000", 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7996e-05 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9357 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.811671319, 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032057078000000003, 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9357 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597091+0000", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597090+0000", 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597098+0000", 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1489999999999994e-06 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597106+0000", 2024-09-21T14:21:44.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3769999999999998e-06 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597110+0000", 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6709999999999998e-06 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629074+0000", 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031963250999999998 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629111+0000", 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7855000000000003e-05 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629148+0000", 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6171999999999999e-05 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:9358 3.14 3:2ed77f83:::rbd_data.11214feba615.0000000000000048:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.465 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.81164689, 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032116296000000003, 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 9358 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:44.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597115+0000", 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597117+0000", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1919999999999999e-06 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597123+0000", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5359999999999998e-06 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.597128+0000", 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3460000000000004e-06 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629161+0000", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032033211999999998 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629197+0000", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5896999999999999e-05 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.629231+0000", 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4530999999999999e-05 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.469 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10578 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699463181, 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031326523000000002, 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10578 2024-09-21T14:21:44.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709299+0000", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709298+0000", 2024-09-21T14:21:44.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709300+0000", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6869999999999999e-06 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709305+0000", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0289999999999996e-06 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709309+0000", 2024-09-21T14:21:44.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.16e-06 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740437+0000", 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031127598999999999 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740525+0000", 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8591000000000006e-05 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740625+0000", 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9724999999999998e-05 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10580 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699448774, 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031423999000000001, 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10580 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709313+0000", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709315+0000", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0059999999999999e-06 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709320+0000", 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3499999999999996e-06 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709325+0000", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4190000000000004e-06 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740638+0000", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031313682000000002 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740687+0000", 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9196999999999997e-05 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740737+0000", 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9675000000000001e-05 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10585 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699434771, 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031620403999999998, 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.478 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10585 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709327+0000", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709328+0000", 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709331+0000", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.954e-06 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709333+0000", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2570000000000002e-06 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740768+0000", 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031434877999999999 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740855+0000", 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7128999999999999e-05 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740948+0000", 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2039000000000004e-05 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10587 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.482 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.69942634, 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031726081000000003, 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10587 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.483 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709335+0000", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709336+0000", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0899999999999999e-06 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709339+0000", 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7760000000000002e-06 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709342+0000", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.351e-06 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.740960+0000", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031618865000000003 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741012+0000", 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1249999999999999e-05 2024-09-21T14:21:44.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741062+0000", 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9950000000000001e-05 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10591 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699416851000001, 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031949914000000003, 2024-09-21T14:21:44.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10591 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709345+0000", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709349+0000", 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4710000000000001e-06 2024-09-21T14:21:44.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709352+0000", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1470000000000001e-06 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709356+0000", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0639999999999998e-06 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741097+0000", 2024-09-21T14:21:44.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031741129 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741185+0000", 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8671000000000005e-05 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741295+0000", 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109634 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10593 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699404054, 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032050040000000002, 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10593 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709358+0000", 2024-09-21T14:21:44.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709359+0000", 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.232e-06 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709362+0000", 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7580000000000001e-06 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709367+0000", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9220000000000002e-06 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741319+0000", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031951949 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741370+0000", 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.075e-05 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741408+0000", 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7667999999999999e-05 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:10596 3.13 3:cd94ea6f:::rbd_data.1121cf46bdbd.0000000000000013:7 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.699365085, 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032199647999999997, 2024-09-21T14:21:44.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 10596 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709397+0000", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709396+0000", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709398+0000", 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7150000000000001e-06 2024-09-21T14:21:44.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709402+0000", 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3500000000000001e-06 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.709404+0000", 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4329999999999998e-06 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:44.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741437+0000", 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032033461999999999 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741516+0000", 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8345999999999998e-05 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.741596+0000", 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0650999999999997e-05 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:44.501 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:44.501 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:44.501 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:44.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:44.503 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:44.504 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T14:21:44.504 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:44.504 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:44.505 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:44.505 INFO:teuthology.orchestra.run.smithi073.stdout:2159 read 0x2f40a5 thru 0x2fae50 (0x6dac bytes) 2024-09-21T14:21:44.505 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:44.505 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:44.505 INFO:teuthology.orchestra.run.smithi073.stdout:2160 trunc from 0x12e94ee to 0x8867dc1 2024-09-21T14:21:44.506 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:44.502+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:44.509 INFO:teuthology.orchestra.run.smithi073.stdout:2161 punch from 0x367ea1 to 0x374c60, (0xcdbf bytes) 2024-09-21T14:21:44.511 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:44.511 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:44.520 INFO:teuthology.orchestra.run.smithi073.stdout:2162 write 0xc83ab42 thru 0xc84565d (0xab1c bytes) 2024-09-21T14:21:44.526 INFO:teuthology.orchestra.run.smithi073.stdout:2163 write 0x8f58c94 thru 0x8f6225f (0x95cc bytes) 2024-09-21T14:21:44.531 INFO:teuthology.orchestra.run.smithi073.stdout:2164 read 0x7f30084 thru 0x7f3f35e (0xf2db bytes) 2024-09-21T14:21:44.531 INFO:teuthology.orchestra.run.smithi073.stdout:2165 write 0xc6bf228 thru 0xc6c912f (0x9f08 bytes) 2024-09-21T14:21:44.533 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:44.533 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:44.537 INFO:teuthology.orchestra.run.smithi073.stdout:2167 write 0xaf880d thru 0xb032cc (0xaac0 bytes) 2024-09-21T14:21:44.556 INFO:teuthology.orchestra.run.smithi073.stdout:2168 read 0x13a307c thru 0x13b2c2b (0xfbb0 bytes) 2024-09-21T14:21:44.556 INFO:teuthology.orchestra.run.smithi073.stdout:2169 write 0x4bb66d2 thru 0x4bc28a1 (0xc1d0 bytes) 2024-09-21T14:21:44.566 INFO:teuthology.orchestra.run.smithi073.stdout:2171 read 0x8f29253 thru 0x8f367c8 (0xd576 bytes) 2024-09-21T14:21:44.566 INFO:teuthology.orchestra.run.smithi073.stdout:2172 read 0x86f1fd9 thru 0x86f7998 (0x59c0 bytes) 2024-09-21T14:21:44.566 INFO:teuthology.orchestra.run.smithi073.stdout:2175 write 0x9071336 thru 0x9077983 (0x664e bytes) 2024-09-21T14:21:44.572 INFO:teuthology.orchestra.run.smithi073.stdout:2176 write 0xace4bf9 thru 0xacf175a (0xcb62 bytes) 2024-09-21T14:21:44.583 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:44.583 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:44.583 INFO:teuthology.orchestra.run.smithi073.stdout:2179 write 0xbcf098f thru 0xbcf3c8c (0x32fe bytes) 2024-09-21T14:21:44.586 INFO:teuthology.orchestra.run.smithi073.stdout:2180 write 0x57cf0e thru 0x584c56 (0x7d49 bytes) 2024-09-21T14:21:44.603 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:44.603+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:44.604 INFO:teuthology.orchestra.run.smithi073.stdout:2183 read 0x5aa85a8 thru 0x5ab1983 (0x93dc bytes) 2024-09-21T14:21:44.607 INFO:teuthology.orchestra.run.smithi073.stdout:2184 trunc from 0xc84565e to 0x9eb555d 2024-09-21T14:21:44.616 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:44.616 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:44.621 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:44.622 INFO:teuthology.orchestra.run.smithi073.stdout:2188 punch from 0x600d161 to 0x600f87e, (0x271d bytes) 2024-09-21T14:21:44.622 INFO:teuthology.orchestra.run.smithi073.stdout:2190 punch from 0x5a68f73 to 0x5a73a69, (0xaaf6 bytes) 2024-09-21T14:21:44.622 INFO:teuthology.orchestra.run.smithi073.stdout:2191 read 0x14f182c thru 0x14f5a9a (0x426f bytes) 2024-09-21T14:21:44.622 INFO:teuthology.orchestra.run.smithi073.stdout:2192 writesame 0x1af0406 thru 0x1afe676 data_size 0x3(0xe271 bytes) 2024-09-21T14:21:44.625 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:44.638 INFO:teuthology.orchestra.run.smithi073.stdout:2194 read 0x1952c2c thru 0x1959823 (0x6bf8 bytes) 2024-09-21T14:21:44.647 INFO:teuthology.orchestra.run.smithi073.stdout:2196 writesame 0xaa79f19 thru 0xaa8643e data_size 0x67(0xc526 bytes) 2024-09-21T14:21:44.647 INFO:teuthology.orchestra.run.smithi073.stdout:2197 write 0x1c3970f thru 0x1c42cd5 (0x95c7 bytes) 2024-09-21T14:21:44.661 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:44.671 INFO:teuthology.orchestra.run.smithi073.stdout:2201 writesame 0xdfb0147 thru 0xdfb29f7 data_size 0x1(0x28b1 bytes) 2024-09-21T14:21:44.671 INFO:teuthology.orchestra.run.smithi073.stdout:2204 read 0xc77e667 thru 0xc78a1f6 (0xbb90 bytes) 2024-09-21T14:21:44.671 INFO:teuthology.orchestra.run.smithi073.stdout:2205 read 0x5d2c01a thru 0x5d2cdcf (0xdb6 bytes) 2024-09-21T14:21:44.671 INFO:teuthology.orchestra.run.smithi073.stdout:2207 writesame 0x1306a38 thru 0x1312c90 data_size 0x1(0xc259 bytes) 2024-09-21T14:21:44.688 INFO:teuthology.orchestra.run.smithi073.stdout:2209 write 0xcebb72a thru 0xcebf423 (0x3cfa bytes) 2024-09-21T14:21:44.694 INFO:teuthology.orchestra.run.smithi073.stdout:2210 writesame 0x5817205 thru 0x581e10b data_size 0x1(0x6f07 bytes) 2024-09-21T14:21:44.706 INFO:teuthology.orchestra.run.smithi073.stdout:2212 writesame 0x55d09cc thru 0x55d3c17 data_size 0x57(0x324c bytes) 2024-09-21T14:21:44.707 INFO:teuthology.orchestra.run.smithi073.stdout:2215 writesame 0x6ccaadf thru 0x6cd0cf7 data_size 0xb(0x6219 bytes) 2024-09-21T14:21:44.708 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:44.709+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:44.712 INFO:teuthology.orchestra.run.smithi073.stdout:2216 punch from 0xd6df635 to 0xd6e2333, (0x2cfe bytes) 2024-09-21T14:21:44.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:44.722 INFO:teuthology.orchestra.run.smithi073.stdout:2220 trunc from 0xdfb29f8 to 0x5e82c7a 2024-09-21T14:21:44.732 INFO:teuthology.orchestra.run.smithi073.stdout:2221 read 0x2d7b72e thru 0x2d84781 (0x9054 bytes) 2024-09-21T14:21:44.732 INFO:teuthology.orchestra.run.smithi073.stdout:2223 read 0x1534156 thru 0x15346d3 (0x57e bytes) 2024-09-21T14:21:44.732 INFO:teuthology.orchestra.run.smithi073.stdout:2224 read 0x3076218 thru 0x307e1df (0x7fc8 bytes) 2024-09-21T14:21:44.738 INFO:teuthology.orchestra.run.smithi073.stdout:2225 trunc from 0x5e82c7a to 0xaae838a 2024-09-21T14:21:44.743 INFO:teuthology.orchestra.run.smithi073.stdout:2226 punch from 0xa38c403 to 0xa39a0b5, (0xdcb2 bytes) 2024-09-21T14:21:44.743 INFO:teuthology.orchestra.run.smithi073.stdout:2227 read 0x4f35f79 thru 0x4f37389 (0x1411 bytes) 2024-09-21T14:21:44.743 INFO:teuthology.orchestra.run.smithi073.stdout:2228 read 0x80175b9 thru 0x802395a (0xc3a2 bytes) 2024-09-21T14:21:44.743 INFO:teuthology.orchestra.run.smithi073.stdout:2229 clone 17 order 19 su 65536 sc 5 2024-09-21T14:21:44.769 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:44.769 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:44.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:44.804 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:44.805+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:44.906 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:44.906+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:44.921 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:44.921 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:44.929 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:45.006 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:45.006+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:45.031 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.031 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:45.123 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:45.124+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:45.183 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.183 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.192 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:45.233 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:45.233+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:45.301 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:21:45.309 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:45.309+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:45.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:21:45.336 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.336 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.345 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:45.368 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone16 from 0xaae838a (overlap 0x12e94ee) to 0x79085f 2024-09-21T14:21:45.410 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:45.410+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:45.455 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.455 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.464 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T14:21:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:2231 writesame 0x571b853 thru 0x571fb03 data_size 0x3(0x42b1 bytes) 2024-09-21T14:21:45.544 INFO:teuthology.orchestra.run.smithi073.stdout:2232 writesame 0xec41891 thru 0xec4b2b5 data_size 0x1(0x9a25 bytes) 2024-09-21T14:21:45.544 INFO:teuthology.orchestra.run.smithi073.stdout:2233 write 0x610c5f3 thru 0x6113b3e (0x754c bytes) 2024-09-21T14:21:45.547 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:45.547+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:45.548 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.553 INFO:teuthology.orchestra.run.smithi073.stdout:2236 trunc from 0xec4b2b6 to 0xa376f4e 2024-09-21T14:21:45.568 INFO:teuthology.orchestra.run.smithi073.stdout:2237 read 0x8f9e394 thru 0x8fa72a7 (0x8f14 bytes) 2024-09-21T14:21:45.575 INFO:teuthology.orchestra.run.smithi073.stdout:2239 write 0xb1aaa24 thru 0xb1b7f25 (0xd502 bytes) 2024-09-21T14:21:45.589 INFO:teuthology.orchestra.run.smithi073.stdout:2240 writesame 0xe73cd32 thru 0xe74b64f data_size 0x35(0xe91e bytes) 2024-09-21T14:21:45.589 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.589 INFO:teuthology.orchestra.run.smithi073.stdout:2243 writesame 0x6bd8132 thru 0x6be6904 data_size 0x1(0xe7d3 bytes) 2024-09-21T14:21:45.605 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.606 INFO:teuthology.orchestra.run.smithi073.stdout:2245 read 0x9a60657 thru 0x9a6c91e (0xc2c8 bytes) 2024-09-21T14:21:45.606 INFO:teuthology.orchestra.run.smithi073.stdout:2247 read 0x17e3b7a thru 0x17f276f (0xebf6 bytes) 2024-09-21T14:21:45.606 INFO:teuthology.orchestra.run.smithi073.stdout:2248 writesame 0xe5c98af thru 0xe5d303b data_size 0x1(0x978d bytes) 2024-09-21T14:21:45.611 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:45.612+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:45.620 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.621 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:45.632 INFO:teuthology.orchestra.run.smithi073.stdout:2249 writesame 0x6f323a5 thru 0x6f34f67 data_size 0x1(0x2bc3 bytes) 2024-09-21T14:21:45.638 INFO:teuthology.orchestra.run.smithi073.stdout:2250 read 0xa96cac9 thru 0xa97a9ca (0xdf02 bytes) 2024-09-21T14:21:45.638 INFO:teuthology.orchestra.run.smithi073.stdout:2251 write 0xb4286e2 thru 0xb43307a (0xa999 bytes) 2024-09-21T14:21:45.641 INFO:teuthology.orchestra.run.smithi073.stdout:2252 read 0x1327371 thru 0x132d2c0 (0x5f50 bytes) 2024-09-21T14:21:45.642 INFO:teuthology.orchestra.run.smithi073.stdout:2253 write 0xf00ec8 thru 0xf043b3 (0x34ec bytes) 2024-09-21T14:21:45.662 INFO:teuthology.orchestra.run.smithi073.stdout:2254 trunc from 0xe74b650 to 0x79cea4 2024-09-21T14:21:45.698 INFO:teuthology.orchestra.run.smithi073.stdout:2255 writesame 0x7ee99c6 thru 0x7ef4753 data_size 0xf(0xad8e bytes) 2024-09-21T14:21:45.698 INFO:teuthology.orchestra.run.smithi073.stdout:2256 read 0x3d4d83a thru 0x3d58a88 (0xb24f bytes) 2024-09-21T14:21:45.708 INFO:teuthology.orchestra.run.smithi073.stdout:2258 writesame 0xc928b3e thru 0xc929d22 data_size 0x3(0x11e5 bytes) 2024-09-21T14:21:45.712 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:45.712+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:45.716 INFO:teuthology.orchestra.run.smithi073.stdout:2262 trunc from 0xc929d23 to 0x60d7175 2024-09-21T14:21:45.720 INFO:teuthology.orchestra.run.smithi073.stdout:2263 write 0x2ec0a37 thru 0x2ece633 (0xdbfd bytes) 2024-09-21T14:21:45.724 INFO:teuthology.orchestra.run.smithi073.stdout:2265 read 0x2edfce0 thru 0x2ee8162 (0x8483 bytes) 2024-09-21T14:21:45.724 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.724 INFO:teuthology.orchestra.run.smithi073.stdout:2267 read 0x412682b thru 0x412aa84 (0x425a bytes) 2024-09-21T14:21:45.724 INFO:teuthology.orchestra.run.smithi073.stdout:2269 read 0x1ea4bf6 thru 0x1eb16f3 (0xcafe bytes) 2024-09-21T14:21:45.728 INFO:teuthology.orchestra.run.smithi073.stdout:2271 write 0x6741066 thru 0x675007e (0xf019 bytes) 2024-09-21T14:21:45.732 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.732 INFO:teuthology.orchestra.run.smithi073.stdout:2273 read 0x3443fa thru 0x34c970 (0x8577 bytes) 2024-09-21T14:21:45.732 INFO:teuthology.orchestra.run.smithi073.stdout:2274 read 0x5935ebf thru 0x593ff13 (0xa055 bytes) 2024-09-21T14:21:45.734 INFO:teuthology.orchestra.run.smithi073.stdout:2275 trunc from 0x675007f to 0x7c60bde 2024-09-21T14:21:45.747 INFO:teuthology.orchestra.run.smithi073.stdout:2276 write 0xa98a498 thru 0xa995ad9 (0xb642 bytes) 2024-09-21T14:21:45.751 INFO:teuthology.orchestra.run.smithi073.stdout:2277 read 0x6ba471 thru 0x6bb4b0 (0x1040 bytes) 2024-09-21T14:21:45.751 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.751 INFO:teuthology.orchestra.run.smithi073.stdout:2280 punch from 0xeb6a92 to 0xeb7f42, (0x14b0 bytes) 2024-09-21T14:21:45.751 INFO:teuthology.orchestra.run.smithi073.stdout:2283 read 0x91d1c43 thru 0x91dc927 (0xace5 bytes) 2024-09-21T14:21:45.757 INFO:teuthology.orchestra.run.smithi073.stdout:2285 write 0xc5ad07b thru 0xc5b478f (0x7715 bytes) 2024-09-21T14:21:45.761 INFO:teuthology.orchestra.run.smithi073.stdout:2287 read 0x52efbff thru 0x52f82e6 (0x86e8 bytes) 2024-09-21T14:21:45.761 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.761 INFO:teuthology.orchestra.run.smithi073.stdout:2289 write 0x250fe89 thru 0x251ecb1 (0xee29 bytes) 2024-09-21T14:21:45.765 INFO:teuthology.orchestra.run.smithi073.stdout:2290 read 0x205f272 thru 0x206d809 (0xe598 bytes) 2024-09-21T14:21:45.765 INFO:teuthology.orchestra.run.smithi073.stdout:2291 write 0x4a461aa thru 0x4a504c1 (0xa318 bytes) 2024-09-21T14:21:45.768 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:45.775 INFO:teuthology.orchestra.run.smithi073.stdout:2294 writesame 0xda8a4fe thru 0xda9599a data_size 0x1(0xb49d bytes) 2024-09-21T14:21:45.775 INFO:teuthology.orchestra.run.smithi073.stdout:2295 write 0xc3589aa thru 0xc35af84 (0x25db bytes) 2024-09-21T14:21:45.778 INFO:teuthology.orchestra.run.smithi073.stdout:2296 read 0xc13b64e thru 0xc13e6db (0x308e bytes) 2024-09-21T14:21:45.778 INFO:teuthology.orchestra.run.smithi073.stdout:2298 punch from 0xa8d26a1 to 0xa8dc024, (0x9983 bytes) 2024-09-21T14:21:45.778 INFO:teuthology.orchestra.run.smithi073.stdout:2299 writesame 0xab714e8 thru 0xab7fdf1 data_size 0xa3(0xe90a bytes) 2024-09-21T14:21:45.780 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.780 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.781 INFO:teuthology.orchestra.run.smithi073.stdout:2304 punch from 0xd539489 to 0xd539619, (0x190 bytes) 2024-09-21T14:21:45.781 INFO:teuthology.orchestra.run.smithi073.stdout:2305 writesame 0x94f6b91 thru 0x94f85f9 data_size 0x1(0x1a69 bytes) 2024-09-21T14:21:45.786 INFO:teuthology.orchestra.run.smithi073.stdout:2306 read 0xd0723b8 thru 0xd074e4a (0x2a93 bytes) 2024-09-21T14:21:45.786 INFO:teuthology.orchestra.run.smithi073.stdout:2307 write 0x7f674cf thru 0x7f6a632 (0x3164 bytes) 2024-09-21T14:21:45.789 INFO:teuthology.orchestra.run.smithi073.stdout:2308 write 0xa55474a thru 0xa562525 (0xdddc bytes) 2024-09-21T14:21:45.790 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:45.792 INFO:teuthology.orchestra.run.smithi073.stdout:2311 write 0xd7dac thru 0xdfe38 (0x808d bytes) 2024-09-21T14:21:45.800 INFO:teuthology.orchestra.run.smithi073.stdout:2312 punch from 0xbf3a7ce to 0xbf3c82b, (0x205d bytes) 2024-09-21T14:21:45.800 INFO:teuthology.orchestra.run.smithi073.stdout:2315 read 0x9b90da0 thru 0x9ba0c2e (0xfe8f bytes) 2024-09-21T14:21:45.800 INFO:teuthology.orchestra.run.smithi073.stdout:2316 punch from 0xb75d407 to 0xb7685c7, (0xb1c0 bytes) 2024-09-21T14:21:45.804 INFO:teuthology.orchestra.run.smithi073.stdout:2317 write 0xe12906f thru 0xe12b87c (0x280e bytes) 2024-09-21T14:21:45.807 INFO:teuthology.orchestra.run.smithi073.stdout:2318 punch from 0xcbded39 to 0xcbe65bb, (0x7882 bytes) 2024-09-21T14:21:45.807 INFO:teuthology.orchestra.run.smithi073.stdout:2320 write 0xdedf6a1 thru 0xdee0670 (0xfd0 bytes) 2024-09-21T14:21:45.811 INFO:teuthology.orchestra.run.smithi073.stdout:2321 read 0x4625317 thru 0x462ab10 (0x57fa bytes) 2024-09-21T14:21:45.811 INFO:teuthology.orchestra.run.smithi073.stdout:2322 clone 18 order 19 su 2048 sc 5 2024-09-21T14:21:45.813 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:45.813+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:45.893 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:45.893 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:45.901 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:45.914 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:45.914+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:46.004 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:46.004 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:46.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:46.015 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:46.015+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:46.115 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:46.115 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:46.115 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:46.116+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:46.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:46.216 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:46.216+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:46.227 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:46.227 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:46.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:21:46.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:21:46.317 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:46.317+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:46.328 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:21:46.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:46.395 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone17 from 0xe12b87d (overlap 0x79cea4) to 0x185c87 2024-09-21T14:21:46.397 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:46.397 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:46.412 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:46.418 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:46.418+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:46.519 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:46.519+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:46.544 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T14:21:46.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:46.569 INFO:teuthology.orchestra.run.smithi073.stdout:2325 write 0xe2139c8 thru 0xe213e60 (0x499 bytes) 2024-09-21T14:21:46.574 INFO:teuthology.orchestra.run.smithi073.stdout:2326 write 0xdf7c9b3 thru 0xdf82535 (0x5b83 bytes) 2024-09-21T14:21:46.577 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:46.577 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:46.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:46.623 INFO:teuthology.orchestra.run.smithi073.stdout:2327 read 0x9ab5f7e thru 0x9ab714d (0x11d0 bytes) 2024-09-21T14:21:46.623 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.623 INFO:teuthology.orchestra.run.smithi073.stdout:2329 write 0xcaff5b3 thru 0xcb05c9d (0x66eb bytes) 2024-09-21T14:21:46.624 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:46.623+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:46.632 INFO:teuthology.orchestra.run.smithi073.stdout:2330 read 0x871d0b thru 0x87b040 (0x9336 bytes) 2024-09-21T14:21:46.635 INFO:teuthology.orchestra.run.smithi073.stdout:2331 punch from 0xcfe8c6 to 0xd09c57, (0xb391 bytes) 2024-09-21T14:21:46.635 INFO:teuthology.orchestra.run.smithi073.stdout:2332 punch from 0x8ee6f88 to 0x8ee9403, (0x247b bytes) 2024-09-21T14:21:46.635 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.639 INFO:teuthology.orchestra.run.smithi073.stdout:2334 trunc from 0xe213e61 to 0xaa16e6d 2024-09-21T14:21:46.698 INFO:teuthology.orchestra.run.smithi073.stdout:2335 read 0x7c133fc thru 0x7c22d7d (0xf982 bytes) 2024-09-21T14:21:46.698 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.698 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.698 INFO:teuthology.orchestra.run.smithi073.stdout:2339 punch from 0x20bbd22 to 0x20c0577, (0x4855 bytes) 2024-09-21T14:21:46.698 INFO:teuthology.orchestra.run.smithi073.stdout:2340 writesame 0x57c1f14 thru 0x57cf8ac data_size 0xd(0xd999 bytes) 2024-09-21T14:21:46.709 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:46.709 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:46.711 INFO:teuthology.orchestra.run.smithi073.stdout:2341 read 0x3e9cdee thru 0x3eaa5bd (0xd7d0 bytes) 2024-09-21T14:21:46.715 INFO:teuthology.orchestra.run.smithi073.stdout:2343 trunc from 0xaa16e6d to 0x7d22b4e 2024-09-21T14:21:46.721 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:46.721+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:46.731 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:46.731 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:46.748 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:46.753 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:46.753 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:46.758 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:46.758 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:46.762 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-21T14:21:46.424450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353968397,"num_objects":687,"num_object_clones":138,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":687,"num_whiteouts":322,"num_read":48981,"num_read_kb":252580,"num_write":7010,"num_write_kb":95530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5578,"ondisk_log_size":5578,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276944,"kb_used_data":15640,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754697776,"statfs":{"total":773094113280,"available":772810522624,"internally_reserved":0,"allocated":16015360,"data_stored":103226496,"data_compressed":5515441,"data_compressed_allocated":7016448,"data_compressed_original":97454568,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":95069703,"num_objects":120,"num_object_clones":39,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":30,"num_read":9215,"num_read_kb":102133,"num_write":1995,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000612"},"pg_stats":[{"pgid":"3.1f","version":"40'90","reported_seq":1123,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.010995+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:42.010995+0000","last_peered":"2024-09-21T14:21:42.010995+0000","last_clean":"2024-09-21T14:21:42.010995+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:42.010995+0000","last_undegraded":"2024-09-21T14:21:42.010995+0000","last_fullsized":"2024-09-21T14:21:42.010995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":10955242,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1029,"num_read_kb":6273,"num_write":76,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"40'60","reported_seq":155,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.831440+0000","last_change":"2024-09-21T14:21:20.149718+0000","last_active":"2024-09-21T14:21:41.831440+0000","last_peered":"2024-09-21T14:21:41.831440+0000","last_clean":"2024-09-21T14:21:41.831440+0000","last_became_active":"2024-09-21T14:21:18.152993+0000","last_became_peered":"2024-09-21T14:21:18.152993+0000","last_unstale":"2024-09-21T14:21:41.831440+0000","last_undegraded":"2024-09-21T14:21:41.831440+0000","last_fullsized":"2024-09-21T14:21:41.831440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:18.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":7233175,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":87,"num_read_kb":4862,"num_write":52,"num_write_kb":2472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"40'73","reported_seq":289,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.816230+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:41.816230+0000","last_peered":"2024-09-21T14:21:41.816230+0000","last_clean":"2024-09-21T14:21:41.816230+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:41.816230+0000","last_undegraded":"2024-09-21T14:21:41.816230+0000","last_fullsized":"2024-09-21T14:21:41.816230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":1877286,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":195,"num_read_kb":288,"num_write":112,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"41'108","reported_seq":1284,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.329523+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:42.329523+0000","last_peered":"2024-09-21T14:21:42.329523+0000","last_clean":"2024-09-21T14:21:42.329523+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:42.329523+0000","last_undegraded":"2024-09-21T14:21:42.329523+0000","last_fullsized":"2024-09-21T14:21:42.329523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":13427163,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":1216,"num_read_kb":20601,"num_write":103,"num_write_kb":2517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"40'142","reported_seq":1257,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.841631+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:41.841631+0000","last_peered":"2024-09-21T14:21:41.841631+0000","last_clean":"2024-09-21T14:21:41.841631+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:41.841631+0000","last_undegraded":"2024-09-21T14:21:41.841631+0000","last_fullsized":"2024-09-21T14:21:41.841631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":17306312,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":1462,"num_read_kb":20199,"num_write":117,"num_write_kb":8860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"40'131","reported_seq":1555,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.431898+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:42.431898+0000","last_peered":"2024-09-21T14:21:42.431898+0000","last_clean":"2024-09-21T14:21:42.431898+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:42.431898+0000","last_undegraded":"2024-09-21T14:21:42.431898+0000","last_fullsized":"2024-09-21T14:21:42.431898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":4505833,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":1612,"num_read_kb":1563,"num_write":147,"num_write_kb":564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"40'113","reported_seq":3027,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.000335+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:42.000335+0000","last_peered":"2024-09-21T14:21:42.000335+0000","last_clean":"2024-09-21T14:21:42.000335+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:42.000335+0000","last_undegraded":"2024-09-21T14:21:42.000335+0000","last_fullsized":"2024-09-21T14:21:42.000335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":7910157,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":3399,"num_read_kb":15856,"num_write":98,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"40'75","reported_seq":686,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.813304+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:41.813304+0000","last_peered":"2024-09-21T14:21:41.813304+0000","last_clean":"2024-09-21T14:21:41.813304+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:41.813304+0000","last_undegraded":"2024-09-21T14:21:41.813304+0000","last_fullsized":"2024-09-21T14:21:41.813304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":3001712,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":595,"num_read_kb":1851,"num_write":66,"num_write_kb":1041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"41'272","reported_seq":487,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.334790+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:42.334790+0000","last_peered":"2024-09-21T14:21:42.334790+0000","last_clean":"2024-09-21T14:21:42.334790+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:42.334790+0000","last_undegraded":"2024-09-21T14:21:42.334790+0000","last_fullsized":"2024-09-21T14:21:42.334790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":8647121,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":1134,"num_read_kb":4009,"num_write":396,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"40'77","reported_seq":774,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979777+0000","last_change":"2024-09-21T14:21:20.755882+0000","last_active":"2024-09-21T14:21:41.979777+0000","last_peered":"2024-09-21T14:21:41.979777+0000","last_clean":"2024-09-21T14:21:41.979777+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:41.979777+0000","last_undegraded":"2024-09-21T14:21:41.979777+0000","last_fullsized":"2024-09-21T14:21:41.979777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:17.531857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":1930092,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":9,"num_read":689,"num_read_kb":869,"num_write":68,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"40'313","reported_seq":1324,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.324553+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:42.324553+0000","last_peered":"2024-09-21T14:21:42.324553+0000","last_clean":"2024-09-21T14:21:42.324553+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:42.324553+0000","last_undegraded":"2024-09-21T14:21:42.324553+0000","last_fullsized":"2024-09-21T14:21:42.324553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":19553046,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2525,"num_read_kb":17104,"num_write":423,"num_write_kb":5372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"40'78","reported_seq":549,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979396+0000","last_change":"2024-09-21T14:21:20.161641+0000","last_active":"2024-09-21T14:21:41.979396+0000","last_peered":"2024-09-21T14:21:41.979396+0000","last_clean":"2024-09-21T14:21:41.979396+0000","last_became_active":"2024-09-21T14:21:18.152393+0000","last_became_peered":"2024-09-21T14:21:18.152393+0000","last_unstale":"2024-09-21T14:21:41.979396+0000","last_undegraded":"2024-09-21T14:21:41.979396+0000","last_fullsized":"2024-09-21T14:21:41.979396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:55.413999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":3507843,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":470,"num_read_kb":888,"num_write":68,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.275249+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:41.275249+0000","last_peered":"2024-09-21T14:21:41.275249+0000","last_clean":"2024-09-21T14:21:41.275249+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:41.275249+0000","last_undegraded":"2024-09-21T14:21:41.275249+0000","last_fullsized":"2024-09-21T14:21:41.275249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'140","reported_seq":795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.010699+0000","last_change":"2024-09-21T14:21:20.755817+0000","last_active":"2024-09-21T14:21:42.010699+0000","last_peered":"2024-09-21T14:21:42.010699+0000","last_clean":"2024-09-21T14:21:42.010699+0000","last_became_active":"2024-09-21T14:21:18.150612+0000","last_became_peered":"2024-09-21T14:21:18.150612+0000","last_unstale":"2024-09-21T14:21:42.010699+0000","last_undegraded":"2024-09-21T14:21:42.010699+0000","last_fullsized":"2024-09-21T14:21:42.010699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:50.934485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":4351738,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":1303,"num_read_kb":1729,"num_write":123,"num_write_kb":742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279735+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:41.279735+0000","last_peered":"2024-09-21T14:21:41.279735+0000","last_clean":"2024-09-21T14:21:41.279735+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:41.279735+0000","last_undegraded":"2024-09-21T14:21:41.279735+0000","last_fullsized":"2024-09-21T14:21:41.279735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"40'114","reported_seq":1490,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.910319+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:41.910319+0000","last_peered":"2024-09-21T14:21:41.910319+0000","last_clean":"2024-09-21T14:21:41.910319+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:41.910319+0000","last_undegraded":"2024-09-21T14:21:41.910319+0000","last_fullsized":"2024-09-21T14:21:41.910319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":4521871,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":1650,"num_read_kb":4222,"num_write":148,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.347434+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:41.347434+0000","last_peered":"2024-09-21T14:21:41.347434+0000","last_clean":"2024-09-21T14:21:41.347434+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:41.347434+0000","last_undegraded":"2024-09-21T14:21:41.347434+0000","last_fullsized":"2024-09-21T14:21:41.347434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"40'103","reported_seq":1376,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.962911+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:41.962911+0000","last_peered":"2024-09-21T14:21:41.962911+0000","last_clean":"2024-09-21T14:21:41.962911+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:41.962911+0000","last_undegraded":"2024-09-21T14:21:41.962911+0000","last_fullsized":"2024-09-21T14:21:41.962911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":16853319,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1267,"num_read_kb":17940,"num_write":90,"num_write_kb":1766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.284737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:41.284737+0000","last_peered":"2024-09-21T14:21:41.284737+0000","last_clean":"2024-09-21T14:21:41.284737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:41.284737+0000","last_undegraded":"2024-09-21T14:21:41.284737+0000","last_fullsized":"2024-09-21T14:21:41.284737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"40'90","reported_seq":3367,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.287793+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:42.287793+0000","last_peered":"2024-09-21T14:21:42.287793+0000","last_clean":"2024-09-21T14:21:42.287793+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:42.287793+0000","last_undegraded":"2024-09-21T14:21:42.287793+0000","last_fullsized":"2024-09-21T14:21:42.287793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":9314384,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":3301,"num_read_kb":24532,"num_write":80,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"40'58","reported_seq":571,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.836763+0000","last_change":"2024-09-21T14:21:20.755906+0000","last_active":"2024-09-21T14:21:41.836763+0000","last_peered":"2024-09-21T14:21:41.836763+0000","last_clean":"2024-09-21T14:21:41.836763+0000","last_became_active":"2024-09-21T14:21:18.151157+0000","last_became_peered":"2024-09-21T14:21:18.151157+0000","last_unstale":"2024-09-21T14:21:41.836763+0000","last_undegraded":"2024-09-21T14:21:41.836763+0000","last_fullsized":"2024-09-21T14:21:41.836763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:25.967224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":7669218,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":497,"num_read_kb":14025,"num_write":64,"num_write_kb":3859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279575+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:41.279575+0000","last_peered":"2024-09-21T14:21:41.279575+0000","last_clean":"2024-09-21T14:21:41.279575+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:41.279575+0000","last_undegraded":"2024-09-21T14:21:41.279575+0000","last_fullsized":"2024-09-21T14:21:41.279575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":41,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.281723+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:41.281723+0000","last_peered":"2024-09-21T14:21:41.281723+0000","last_clean":"2024-09-21T14:21:41.281723+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:41.281723+0000","last_undegraded":"2024-09-21T14:21:41.281723+0000","last_fullsized":"2024-09-21T14:21:41.281723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"40'391","reported_seq":496,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.289294+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:42.289294+0000","last_peered":"2024-09-21T14:21:42.289294+0000","last_clean":"2024-09-21T14:21:42.289294+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:42.289294+0000","last_undegraded":"2024-09-21T14:21:42.289294+0000","last_fullsized":"2024-09-21T14:21:42.289294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":2071353,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":1667,"num_read_kb":1584,"num_write":647,"num_write_kb":1142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":42,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.310690+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:41.310690+0000","last_peered":"2024-09-21T14:21:41.310690+0000","last_clean":"2024-09-21T14:21:41.310690+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:41.310690+0000","last_undegraded":"2024-09-21T14:21:41.310690+0000","last_fullsized":"2024-09-21T14:21:41.310690+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'115","reported_seq":395,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.826338+0000","last_change":"2024-09-21T14:21:20.151933+0000","last_active":"2024-09-21T14:21:41.826338+0000","last_peered":"2024-09-21T14:21:41.826338+0000","last_clean":"2024-09-21T14:21:41.826338+0000","last_became_active":"2024-09-21T14:21:18.147902+0000","last_became_peered":"2024-09-21T14:21:18.147902+0000","last_unstale":"2024-09-21T14:21:41.826338+0000","last_undegraded":"2024-09-21T14:21:41.826338+0000","last_fullsized":"2024-09-21T14:21:41.826338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:09.190450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":3446646,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":365,"num_read_kb":2128,"num_write":122,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279841+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:41.279841+0000","last_peered":"2024-09-21T14:21:41.279841+0000","last_clean":"2024-09-21T14:21:41.279841+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:41.279841+0000","last_undegraded":"2024-09-21T14:21:41.279841+0000","last_fullsized":"2024-09-21T14:21:41.279841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"41'260","reported_seq":1081,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.427869+0000","last_change":"2024-09-21T14:21:20.145954+0000","last_active":"2024-09-21T14:21:42.427869+0000","last_peered":"2024-09-21T14:21:42.427869+0000","last_clean":"2024-09-21T14:21:42.427869+0000","last_became_active":"2024-09-21T14:21:18.152479+0000","last_became_peered":"2024-09-21T14:21:18.152479+0000","last_unstale":"2024-09-21T14:21:42.427869+0000","last_undegraded":"2024-09-21T14:21:42.427869+0000","last_fullsized":"2024-09-21T14:21:42.427869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:27.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":19673601,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":1613,"num_read_kb":2798,"num_write":366,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.275098+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:41.275098+0000","last_peered":"2024-09-21T14:21:41.275098+0000","last_clean":"2024-09-21T14:21:41.275098+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:41.275098+0000","last_undegraded":"2024-09-21T14:21:41.275098+0000","last_fullsized":"2024-09-21T14:21:41.275098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"40'345","reported_seq":1582,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.962912+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:41.962912+0000","last_peered":"2024-09-21T14:21:41.962912+0000","last_clean":"2024-09-21T14:21:41.962912+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:41.962912+0000","last_undegraded":"2024-09-21T14:21:41.962912+0000","last_fullsized":"2024-09-21T14:21:41.962912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":8251443,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":3378,"num_read_kb":7241,"num_write":442,"num_write_kb":2178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"41'268","reported_seq":1769,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.438276+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:42.438276+0000","last_peered":"2024-09-21T14:21:42.438276+0000","last_clean":"2024-09-21T14:21:42.438276+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:42.438276+0000","last_undegraded":"2024-09-21T14:21:42.438276+0000","last_fullsized":"2024-09-21T14:21:42.438276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":57716938,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":3056,"num_read_kb":10847,"num_write":268,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"41'166","reported_seq":293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.346629+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:42.346629+0000","last_peered":"2024-09-21T14:21:42.346629+0000","last_clean":"2024-09-21T14:21:42.346629+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:42.346629+0000","last_undegraded":"2024-09-21T14:21:42.346629+0000","last_fullsized":"2024-09-21T14:21:42.346629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":3207626,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":336,"num_read_kb":1147,"num_write":178,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"40'91","reported_seq":750,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.824589+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:41.824589+0000","last_peered":"2024-09-21T14:21:41.824589+0000","last_clean":"2024-09-21T14:21:41.824589+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:41.824589+0000","last_undegraded":"2024-09-21T14:21:41.824589+0000","last_fullsized":"2024-09-21T14:21:41.824589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":14237900,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":667,"num_read_kb":1613,"num_write":179,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"41'199","reported_seq":332,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.431630+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:42.431630+0000","last_peered":"2024-09-21T14:21:42.431630+0000","last_clean":"2024-09-21T14:21:42.431630+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:42.431630+0000","last_undegraded":"2024-09-21T14:21:42.431630+0000","last_fullsized":"2024-09-21T14:21:42.431630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":6138736,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":664,"num_read_kb":833,"num_write":254,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"40'119","reported_seq":1443,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979690+0000","last_change":"2024-09-21T14:21:34.229265+0000","last_active":"2024-09-21T14:21:41.979690+0000","last_peered":"2024-09-21T14:21:41.979690+0000","last_clean":"2024-09-21T14:21:41.979690+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:21:41.979690+0000","last_undegraded":"2024-09-21T14:21:41.979690+0000","last_fullsized":"2024-09-21T14:21:41.979690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:04.756321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":15783528,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":1088,"num_read_kb":7557,"num_write":115,"num_write_kb":2819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"40'602","reported_seq":1538,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.999967+0000","last_change":"2024-09-21T14:21:20.756021+0000","last_active":"2024-09-21T14:21:41.999967+0000","last_peered":"2024-09-21T14:21:41.999967+0000","last_clean":"2024-09-21T14:21:41.999967+0000","last_became_active":"2024-09-21T14:21:18.152859+0000","last_became_peered":"2024-09-21T14:21:18.152859+0000","last_unstale":"2024-09-21T14:21:41.999967+0000","last_undegraded":"2024-09-21T14:21:41.999967+0000","last_fullsized":"2024-09-21T14:21:41.999967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.438917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":9891736,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":3798,"num_read_kb":9783,"num_write":1020,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"40'140","reported_seq":1815,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.840034+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:41.840034+0000","last_peered":"2024-09-21T14:21:41.840034+0000","last_clean":"2024-09-21T14:21:41.840034+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:41.840034+0000","last_undegraded":"2024-09-21T14:21:41.840034+0000","last_fullsized":"2024-09-21T14:21:41.840034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":16108775,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":1966,"num_read_kb":6775,"num_write":128,"num_write_kb":13529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"40'105","reported_seq":1138,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979380+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:41.979380+0000","last_peered":"2024-09-21T14:21:41.979380+0000","last_clean":"2024-09-21T14:21:41.979380+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:41.979380+0000","last_undegraded":"2024-09-21T14:21:41.979380+0000","last_fullsized":"2024-09-21T14:21:41.979380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":5807379,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":1350,"num_read_kb":7890,"num_write":94,"num_write_kb":2619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"40'134","reported_seq":820,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.818774+0000","last_change":"2024-09-21T14:21:20.160536+0000","last_active":"2024-09-21T14:21:41.818774+0000","last_peered":"2024-09-21T14:21:41.818774+0000","last_clean":"2024-09-21T14:21:41.818774+0000","last_became_active":"2024-09-21T14:21:18.151161+0000","last_became_peered":"2024-09-21T14:21:18.151161+0000","last_unstale":"2024-09-21T14:21:41.818774+0000","last_undegraded":"2024-09-21T14:21:41.818774+0000","last_fullsized":"2024-09-21T14:21:41.818774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:48.604891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":25245999,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":706,"num_read_kb":11107,"num_write":180,"num_write_kb":10542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'116","reported_seq":835,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.317868+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:42.317868+0000","last_peered":"2024-09-21T14:21:42.317868+0000","last_clean":"2024-09-21T14:21:42.317868+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:42.317868+0000","last_undegraded":"2024-09-21T14:21:42.317868+0000","last_fullsized":"2024-09-21T14:21:42.317868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":9966903,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1262,"num_read_kb":8120,"num_write":107,"num_write_kb":2495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"40'370","reported_seq":2621,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979309+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:41.979309+0000","last_peered":"2024-09-21T14:21:41.979309+0000","last_clean":"2024-09-21T14:21:41.979309+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:41.979309+0000","last_undegraded":"2024-09-21T14:21:41.979309+0000","last_fullsized":"2024-09-21T14:21:41.979309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":13263935,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":4518,"num_read_kb":16246,"num_write":518,"num_write_kb":8109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":353378010,"num_objects":683,"num_object_clones":138,"num_object_copies":1366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":683,"num_whiteouts":322,"num_read":48865,"num_read_kb":252480,"num_write":6849,"num_write_kb":93112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12099584,"data_stored":99033782,"data_compressed":5342186,"data_compressed_allocated":6782976,"data_compressed_original":94882280,"omap_allocated":0,"internal_metadata":0},"log_size":5458,"ondisk_log_size":5458,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1245184,"data_stored":5534171,"data_compressed":270018,"data_compressed_allocated":372736,"data_compressed_original":5033984,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2011136,"data_stored":11900474,"data_compressed":637888,"data_compressed_allocated":876544,"data_compressed_original":11148439,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1536000,"data_stored":7938066,"data_compressed":366377,"data_compressed_allocated":516096,"data_compressed_original":7302802,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1986560,"data_stored":14353438,"data_compressed":708202,"data_compressed_allocated":790528,"data_compressed_original":13585730,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37912,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333928,"statfs":{"total":96636764160,"available":96597942272,"internally_reserved":0,"allocated":2768896,"data_stored":18924577,"data_compressed":1030028,"data_compressed_allocated":1363968,"data_compressed_original":17941368,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":2097152,"data_stored":13992528,"data_compressed":788059,"data_compressed_allocated":1081344,"data_compressed_original":13367323,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1728512,"data_stored":9245555,"data_compressed":549388,"data_compressed_allocated":659456,"data_compressed_original":8611335,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37800,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334040,"statfs":{"total":96636764160,"available":96598056960,"internally_reserved":0,"allocated":2641920,"data_stored":21337687,"data_compressed":1165481,"data_compressed_allocated":1355776,"data_compressed_original":20463587,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1978368,"data_stored":19988675,"data_compressed":1072829,"data_compressed_allocated":1228800,"data_compressed_original":19374051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1286144,"data_stored":9046425,"data_compressed":548811,"data_compressed_allocated":659456,"data_compressed_original":8599047,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1667072,"data_stored":13926989,"data_compressed":793900,"data_compressed_allocated":1093632,"data_compressed_original":13490203,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2260992,"data_stored":18142065,"data_compressed":997618,"data_compressed_allocated":1318912,"data_compressed_original":17367928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1552384,"data_stored":14232515,"data_compressed":711291,"data_compressed_allocated":794624,"data_compressed_original":13651266,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":999424,"data_stored":6646958,"data_compressed":309831,"data_compressed_allocated":438272,"data_compressed_original":6217362,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1560576,"data_stored":11708229,"data_compressed":637888,"data_compressed_allocated":876544,"data_compressed_original":11148439,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":794624,"data_stored":5341926,"data_compressed":270018,"data_compressed_allocated":372736,"data_compressed_original":5033984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:46.764 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:2349 write 0x45f5697 thru 0x46012d0 (0xbc3a bytes) 2024-09-21T14:21:46.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:46.777 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:46.777 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:46.778 INFO:teuthology.orchestra.run.smithi073.stdout:2350 write 0xc6a4f1 thru 0xc79ec6 (0xf9d6 bytes) 2024-09-21T14:21:46.790 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.791 INFO:teuthology.orchestra.run.smithi073.stdout:2352 write 0x9ea39a thru 0x9f87c5 (0xe42c bytes) 2024-09-21T14:21:46.799 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:46.799 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:46.818 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:46.818 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:46.822 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:46.822+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:46.838 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:46.838 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:46.844 INFO:teuthology.orchestra.run.smithi073.stdout:2354 trunc from 0x7d22b4e to 0xa66755a 2024-09-21T14:21:46.850 INFO:teuthology.orchestra.run.smithi073.stdout:2356 write 0x1a6c22 thru 0x1a6f3e (0x31d bytes) 2024-09-21T14:21:46.868 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:46.869 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:46.876 INFO:teuthology.orchestra.run.smithi073.stdout:2360 read 0x58c3c7 thru 0x58ff31 (0x3b6b bytes) 2024-09-21T14:21:46.877 INFO:teuthology.orchestra.run.smithi073.stdout:2363 write 0xa408535 thru 0xa416f9d (0xea69 bytes) 2024-09-21T14:21:46.885 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.885 INFO:teuthology.orchestra.run.smithi073.stdout:2365 write 0x2c609f1 thru 0x2c68cb1 (0x82c1 bytes) 2024-09-21T14:21:46.895 INFO:teuthology.orchestra.run.smithi073.stdout:2366 read 0x61a15e5 thru 0x61a5174 (0x3b90 bytes) 2024-09-21T14:21:46.895 INFO:teuthology.orchestra.run.smithi073.stdout:2368 read 0x32d1872 thru 0x32d55e3 (0x3d72 bytes) 2024-09-21T14:21:46.899 INFO:teuthology.orchestra.run.smithi073.stdout:2371 trunc from 0xa66755a to 0xdce2f09 2024-09-21T14:21:46.906 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:46.906 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:46.906 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:46.906 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:46.906 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:46.908 INFO:teuthology.orchestra.run.smithi073.stdout:2372 write 0x2e242b9 thru 0x2e3104e (0xcd96 bytes) 2024-09-21T14:21:46.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:46.949 INFO:teuthology.orchestra.run.smithi073.stdout:2373 read 0x234f43f thru 0x235dc51 (0xe813 bytes) 2024-09-21T14:21:46.949 INFO:teuthology.orchestra.run.smithi073.stdout:2374 writesame 0x3cedd28 thru 0x3cfd7c1 data_size 0x1(0xfa9a bytes) 2024-09-21T14:21:46.950 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:46.949+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:46.972 INFO:teuthology.orchestra.run.smithi073.stdout:2375 write 0xe09296a thru 0xe0993ca (0x6a61 bytes) 2024-09-21T14:21:46.981 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.981 INFO:teuthology.orchestra.run.smithi073.stdout:2378 write 0xa4d0655 thru 0xa4dc6a3 (0xc04f bytes) 2024-09-21T14:21:46.983 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.983 INFO:teuthology.orchestra.run.smithi073.stdout:2381 read 0x3e10f77 thru 0x3e1bca2 (0xad2c bytes) 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:2382 read 0x97248f5 thru 0x9724e9d (0x5a9 bytes) 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:2383 punch from 0x34ae075 to 0x34aff3d, (0x1ec8 bytes) 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:2385 read 0xbdc445d thru 0xbdd2888 (0xe42c bytes) 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:2387 read 0x5404648 thru 0x54126bb (0xe074 bytes) 2024-09-21T14:21:46.984 INFO:teuthology.orchestra.run.smithi073.stdout:2388 write 0x9ca560 thru 0x9d6081 (0xbb22 bytes) 2024-09-21T14:21:46.986 INFO:teuthology.orchestra.run.smithi073.stdout:2389 read 0x4468255 thru 0x446f73a (0x74e6 bytes) 2024-09-21T14:21:46.986 INFO:teuthology.orchestra.run.smithi073.stdout:2390 read 0x8645c59 thru 0x864ce52 (0x71fa bytes) 2024-09-21T14:21:46.986 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:46.987 INFO:teuthology.orchestra.run.smithi073.stdout:2392 read 0x56ebc9c thru 0x56efcca (0x402f bytes) 2024-09-21T14:21:46.988 INFO:teuthology.orchestra.run.smithi073.stdout:2393 trunc from 0xe0993cb to 0xce06fd4 2024-09-21T14:21:47.004 INFO:teuthology.orchestra.run.smithi073.stdout:2394 trunc from 0xce06fd4 to 0x92fc454 2024-09-21T14:21:47.017 INFO:teuthology.orchestra.run.smithi073.stdout:2396 writesame 0x3f654e thru 0x3fc620 data_size 0x1(0x60d3 bytes) 2024-09-21T14:21:47.024 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:47.024+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.86429487, 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41129 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 319488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914611+0000", 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17062123000000001, 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029105128000000001, 2024-09-21T14:21:47.091 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41129 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914611+0000", 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914611+0000", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914610+0000", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914611+0000", 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.097e-06 2024-09-21T14:21:47.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914614+0000", 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9100000000000001e-06 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914617+0000", 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3920000000000001e-06 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943619+0000", 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029002275000000001 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943667+0000", 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7806999999999998e-05 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943716+0000", 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8847999999999997e-05 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.095 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41130 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914619+0000", 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17061310499999999, 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029232201999999999, 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41130 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.096 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914619+0000", 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914619+0000", 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914618+0000", 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914619+0000", 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0189999999999999e-06 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914622+0000", 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7989999999999998e-06 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914625+0000", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4530000000000001e-06 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943739+0000", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029114577999999999 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943789+0000", 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9787999999999998e-05 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943851+0000", 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1772000000000004e-05 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41131 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914627+0000", 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17060512999999999, 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029347286, 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41131 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914627+0000", 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914627+0000", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914626+0000", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914628+0000", 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1930000000000001e-06 2024-09-21T14:21:47.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914631+0000", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9979999999999999e-06 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914633+0000", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3910000000000001e-06 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943875+0000", 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029242364999999999 2024-09-21T14:21:47.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943920+0000", 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4768000000000002e-05 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.943974+0000", 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3773000000000002e-05 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41132 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 354304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914635+0000", 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17059626, 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029479403000000001, 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41132 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914635+0000", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914635+0000", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914635+0000", 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914636+0000", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1340000000000001e-06 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914639+0000", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.926e-06 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914642+0000", 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4150000000000002e-06 2024-09-21T14:21:47.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944005+0000", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029362968999999999 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944054+0000", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9505000000000001e-05 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944115+0000", 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0667e-05 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41133 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 364544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914644+0000", 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17058810399999999, 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029604174, 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41133 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914644+0000", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914644+0000", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914643+0000", 2024-09-21T14:21:47.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914645+0000", 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1459999999999999e-06 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914647+0000", 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7839999999999999e-06 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914650+0000", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5629999999999999e-06 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944141+0000", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029490850999999998 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944191+0000", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0259000000000001e-05 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944248+0000", 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6771999999999998e-05 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41134 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914652+0000", 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17057977499999999, 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029841227000000001, 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41134 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914652+0000", 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914652+0000", 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914652+0000", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914653+0000", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1909999999999999e-06 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914656+0000", 2024-09-21T14:21:47.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8169999999999999e-06 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914658+0000", 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4420000000000001e-06 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944286+0000", 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029627789000000002 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944379+0000", 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3105999999999999e-05 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944493+0000", 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114088 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41135 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 385024~1588] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.117 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914660+0000", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17057151200000001, 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030082806, 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41135 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914660+0000", 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914660+0000", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914660+0000", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914661+0000", 2024-09-21T14:21:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.074e-06 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914664+0000", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.835e-06 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914667+0000", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2760000000000001e-06 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944506+0000", 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839122999999999 2024-09-21T14:21:47.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944553+0000", 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6755000000000002e-05 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944743+0000", 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018994500000000001 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41136 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914669+0000", 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.170562516, 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03027788, 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41136 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914669+0000", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914669+0000", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914669+0000", 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914672+0000", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4599999999999999e-06 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914676+0000", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914678+0000", 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3609999999999999e-06 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944776+0000", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030097678999999999 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944866+0000", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0667000000000005e-05 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944947+0000", 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0741999999999994e-05 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41137 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914681+0000", 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17055124099999999, 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030374251000000001, 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.126 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41137 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914681+0000", 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914681+0000", 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914680+0000", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914681+0000", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1880000000000001e-06 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914684+0000", 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9830000000000001e-06 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914687+0000", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4669999999999998e-06 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.944961+0000", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030273635 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945016+0000", 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5668999999999998e-05 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945055+0000", 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41138 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 419840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914689+0000", 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.170542951, 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030562299000000001, 2024-09-21T14:21:47.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41138 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914689+0000", 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914689+0000", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914689+0000", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914690+0000", 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.153e-06 2024-09-21T14:21:47.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914693+0000", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.802e-06 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914695+0000", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3949999999999999e-06 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945088+0000", 2024-09-21T14:21:47.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030392782 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945174+0000", 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6199000000000006e-05 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945251+0000", 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7171999999999999e-05 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.134 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41139 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914697+0000", 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17053484599999999, 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030647932999999999, 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41139 2024-09-21T14:21:47.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914697+0000", 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914697+0000", 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914697+0000", 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914698+0000", 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1179999999999999e-06 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914701+0000", 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8590000000000001e-06 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914703+0000", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4229999999999999e-06 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945262+0000", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030559399000000001 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945308+0000", 2024-09-21T14:21:47.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5772999999999997e-05 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945345+0000", 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6569000000000002e-05 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41140 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 440320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914705+0000", 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17052677299999999, 2024-09-21T14:21:47.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030837342, 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41140 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914705+0000", 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914705+0000", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914705+0000", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914706+0000", 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.105e-06 2024-09-21T14:21:47.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914709+0000", 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.774e-06 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914711+0000", 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.351e-06 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945376+0000", 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030665095 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945462+0000", 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6360000000000007e-05 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945542+0000", 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9864000000000006e-05 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41141 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914713+0000", 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17051881999999999, 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030920617000000001, 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41141 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914713+0000", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914713+0000", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914713+0000", 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914714+0000", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1510000000000001e-06 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914717+0000", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.852e-06 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914719+0000", 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.249e-06 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945553+0000", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03083433 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945596+0000", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2939999999999999e-05 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945634+0000", 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7299999999999999e-05 2024-09-21T14:21:47.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41142 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914721+0000", 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17051081600000001, 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031109917000000001, 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41142 2024-09-21T14:21:47.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914721+0000", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914721+0000", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914721+0000", 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914722+0000", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.065e-06 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914725+0000", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8880000000000001e-06 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914727+0000", 2024-09-21T14:21:47.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4219999999999999e-06 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945663+0000", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030935771000000001 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945751+0000", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8336000000000003e-05 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945831+0000", 2024-09-21T14:21:47.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9639000000000001e-05 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41143 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914729+0000", 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17050273099999999, 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031195210000000001, 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.152 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41143 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914729+0000", 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914729+0000", 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914729+0000", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914730+0000", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.156e-06 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914733+0000", 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7970000000000001e-06 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914735+0000", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3889999999999999e-06 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945842+0000", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031106958000000001 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945887+0000", 2024-09-21T14:21:47.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5058999999999997e-05 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945924+0000", 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41144 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914737+0000", 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.170494754, 2024-09-21T14:21:47.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031384384000000001, 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41144 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914737+0000", 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914737+0000", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914737+0000", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914738+0000", 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9099999999999991e-07 2024-09-21T14:21:47.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914741+0000", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8109999999999999e-06 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914743+0000", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.446e-06 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.945954+0000", 2024-09-21T14:21:47.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031211381999999999 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946043+0000", 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8887999999999993e-05 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946121+0000", 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8075e-05 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41145 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914745+0000", 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.170486843, 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031475867999999997, 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41145 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914745+0000", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914745+0000", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914745+0000", 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914746+0000", 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914749+0000", 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8270000000000002e-06 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914751+0000", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.491e-06 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946134+0000", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031382659 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946183+0000", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9078e-05 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946221+0000", 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7852999999999998e-05 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41146 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914753+0000", 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.17047855200000001, 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031659949999999999, 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41146 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914753+0000", 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914753+0000", 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914753+0000", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914754+0000", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0440000000000001e-06 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914757+0000", 2024-09-21T14:21:47.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8109999999999999e-06 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914759+0000", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3889999999999999e-06 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946251+0000", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031491502999999997 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946335+0000", 2024-09-21T14:21:47.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4351999999999995e-05 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946413+0000", 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8054999999999997e-05 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:41147 3.1c 3:38d1d03e:::rbd_data.112168cdcfd1.000000000000005d:15 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:46.914761+0000", 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.170470542, 2024-09-21T14:21:47.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031742484000000001, 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 41147 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914761+0000", 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914761+0000", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914761+0000", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914762+0000", 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.083e-06 2024-09-21T14:21:47.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914765+0000", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7539999999999998e-06 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.914767+0000", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3539999999999998e-06 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946423+0000", 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031656146000000003 2024-09-21T14:21:47.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946468+0000", 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4932000000000003e-05 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:46.946504+0000", 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5420999999999998e-05 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.173 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:47.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:47.176 INFO:teuthology.orchestra.run.smithi073.stdout:2397 writesame 0xda2e8dd thru 0xda3c9ee data_size 0xc2(0xe112 bytes) 2024-09-21T14:21:47.176 INFO:teuthology.orchestra.run.smithi073.stdout:2398 write 0x5a8d7a7 thru 0x5a8e690 (0xeea bytes) 2024-09-21T14:21:47.176 INFO:teuthology.orchestra.run.smithi073.stdout:2399 writesame 0x9c75b74 thru 0x9c769fd data_size 0x1(0xe8a bytes) 2024-09-21T14:21:47.176 INFO:teuthology.orchestra.run.smithi073.stdout:2400 read 0xaebf367 thru 0xaecd105 (0xdd9f bytes) 2024-09-21T14:21:47.176 INFO:teuthology.orchestra.run.smithi073.stdout:2402 trunc from 0xda3c9ef to 0x7400a46 2024-09-21T14:21:47.177 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:47.177 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:2404 read 0x4c33c3e thru 0x4c3ec4c (0xb00f bytes) 2024-09-21T14:21:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:2406 read 0x3970484 thru 0x397e7d0 (0xe34d bytes) 2024-09-21T14:21:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:2407 write 0xa1b0185 thru 0xa1bae44 (0xacc0 bytes) 2024-09-21T14:21:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-21T14:21:46.424450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353968397,"num_objects":687,"num_object_clones":138,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":687,"num_whiteouts":322,"num_read":48981,"num_read_kb":252580,"num_write":7010,"num_write_kb":95530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5578,"ondisk_log_size":5578,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276944,"kb_used_data":15640,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754697776,"statfs":{"total":773094113280,"available":772810522624,"internally_reserved":0,"allocated":16015360,"data_stored":103226496,"data_compressed":5515441,"data_compressed_allocated":7016448,"data_compressed_original":97454568,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":95069703,"num_objects":120,"num_object_clones":39,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":30,"num_read":9215,"num_read_kb":102133,"num_write":1995,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000612"},"pg_stats":[{"pgid":"3.1f","version":"40'90","reported_seq":1123,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.010995+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:42.010995+0000","last_peered":"2024-09-21T14:21:42.010995+0000","last_clean":"2024-09-21T14:21:42.010995+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:42.010995+0000","last_undegraded":"2024-09-21T14:21:42.010995+0000","last_fullsized":"2024-09-21T14:21:42.010995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":10955242,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1029,"num_read_kb":6273,"num_write":76,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"40'60","reported_seq":155,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.831440+0000","last_change":"2024-09-21T14:21:20.149718+0000","last_active":"2024-09-21T14:21:41.831440+0000","last_peered":"2024-09-21T14:21:41.831440+0000","last_clean":"2024-09-21T14:21:41.831440+0000","last_became_active":"2024-09-21T14:21:18.152993+0000","last_became_peered":"2024-09-21T14:21:18.152993+0000","last_unstale":"2024-09-21T14:21:41.831440+0000","last_undegraded":"2024-09-21T14:21:41.831440+0000","last_fullsized":"2024-09-21T14:21:41.831440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:18.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":7233175,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":87,"num_read_kb":4862,"num_write":52,"num_write_kb":2472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"40'73","reported_seq":289,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.816230+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:41.816230+0000","last_peered":"2024-09-21T14:21:41.816230+0000","last_clean":"2024-09-21T14:21:41.816230+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:41.816230+0000","last_undegraded":"2024-09-21T14:21:41.816230+0000","last_fullsized":"2024-09-21T14:21:41.816230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":1877286,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":195,"num_read_kb":288,"num_write":112,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"41'108","reported_seq":1284,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.329523+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:42.329523+0000","last_peered":"2024-09-21T14:21:42.329523+0000","last_clean":"2024-09-21T14:21:42.329523+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:42.329523+0000","last_undegraded":"2024-09-21T14:21:42.329523+0000","last_fullsized":"2024-09-21T14:21:42.329523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":13427163,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":1216,"num_read_kb":20601,"num_write":103,"num_write_kb":2517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"40'142","reported_seq":1257,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.841631+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:41.841631+0000","last_peered":"2024-09-21T14:21:41.841631+0000","last_clean":"2024-09-21T14:21:41.841631+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:41.841631+0000","last_undegraded":"2024-09-21T14:21:41.841631+0000","last_fullsized":"2024-09-21T14:21:41.841631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":17306312,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":1462,"num_read_kb":20199,"num_write":117,"num_write_kb":8860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"40'131","reported_seq":1555,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.431898+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:42.431898+0000","last_peered":"2024-09-21T14:21:42.431898+0000","last_clean":"2024-09-21T14:21:42.431898+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:42.431898+0000","last_undegraded":"2024-09-21T14:21:42.431898+0000","last_fullsized":"2024-09-21T14:21:42.431898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":4505833,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":1612,"num_read_kb":1563,"num_write":147,"num_write_kb":564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"40'113","reported_seq":3027,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.000335+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:42.000335+0000","last_peered":"2024-09-21T14:21:42.000335+0000","last_clean":"2024-09-21T14:21:42.000335+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:42.000335+0000","last_undegraded":"2024-09-21T14:21:42.000335+0000","last_fullsized":"2024-09-21T14:21:42.000335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":7910157,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":3399,"num_read_kb":15856,"num_write":98,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"40'75","reported_seq":686,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.813304+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:41.813304+0000","last_peered":"2024-09-21T14:21:41.813304+0000","last_clean":"2024-09-21T14:21:41.813304+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:41.813304+0000","last_undegraded":"2024-09-21T14:21:41.813304+0000","last_fullsized":"2024-09-21T14:21:41.813304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":3001712,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":595,"num_read_kb":1851,"num_write":66,"num_write_kb":1041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"41'272","reported_seq":487,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.334790+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:42.334790+0000","last_peered":"2024-09-21T14:21:42.334790+0000","last_clean":"2024-09-21T14:21:42.334790+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:42.334790+0000","last_undegraded":"2024-09-21T14:21:42.334790+0000","last_fullsized":"2024-09-21T14:21:42.334790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":8647121,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":1134,"num_read_kb":4009,"num_write":396,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"40'77","reported_seq":774,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979777+0000","last_change":"2024-09-21T14:21:20.755882+0000","last_active":"2024-09-21T14:21:41.979777+0000","last_peered":"2024-09-21T14:21:41.979777+0000","last_clean":"2024-09-21T14:21:41.979777+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:41.979777+0000","last_undegraded":"2024-09-21T14:21:41.979777+0000","last_fullsized":"2024-09-21T14:21:41.979777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:17.531857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":1930092,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":9,"num_read":689,"num_read_kb":869,"num_write":68,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"40'313","reported_seq":1324,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.324553+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:42.324553+0000","last_peered":"2024-09-21T14:21:42.324553+0000","last_clean":"2024-09-21T14:21:42.324553+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:42.324553+0000","last_undegraded":"2024-09-21T14:21:42.324553+0000","last_fullsized":"2024-09-21T14:21:42.324553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":19553046,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2525,"num_read_kb":17104,"num_write":423,"num_write_kb":5372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"40'78","reported_seq":549,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979396+0000","last_change":"2024-09-21T14:21:20.161641+0000","last_active":"2024-09-21T14:21:41.979396+0000","last_peered":"2024-09-21T14:21:41.979396+0000","last_clean":"2024-09-21T14:21:41.979396+0000","last_became_active":"2024-09-21T14:21:18.152393+0000","last_became_peered":"2024-09-21T14:21:18.152393+0000","last_unstale":"2024-09-21T14:21:41.979396+0000","last_undegraded":"2024-09-21T14:21:41.979396+0000","last_fullsized":"2024-09-21T14:21:41.979396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:55.413999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":3507843,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":470,"num_read_kb":888,"num_write":68,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.275249+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:41.275249+0000","last_peered":"2024-09-21T14:21:41.275249+0000","last_clean":"2024-09-21T14:21:41.275249+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:41.275249+0000","last_undegraded":"2024-09-21T14:21:41.275249+0000","last_fullsized":"2024-09-21T14:21:41.275249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'140","reported_seq":795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.010699+0000","last_change":"2024-09-21T14:21:20.755817+0000","last_active":"2024-09-21T14:21:42.010699+0000","last_peered":"2024-09-21T14:21:42.010699+0000","last_clean":"2024-09-21T14:21:42.010699+0000","last_became_active":"2024-09-21T14:21:18.150612+0000","last_became_peered":"2024-09-21T14:21:18.150612+0000","last_unstale":"2024-09-21T14:21:42.010699+0000","last_undegraded":"2024-09-21T14:21:42.010699+0000","last_fullsized":"2024-09-21T14:21:42.010699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:50.934485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":4351738,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":1303,"num_read_kb":1729,"num_write":123,"num_write_kb":742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279735+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:41.279735+0000","last_peered":"2024-09-21T14:21:41.279735+0000","last_clean":"2024-09-21T14:21:41.279735+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:41.279735+0000","last_undegraded":"2024-09-21T14:21:41.279735+0000","last_fullsized":"2024-09-21T14:21:41.279735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"40'114","reported_seq":1490,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.910319+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:41.910319+0000","last_peered":"2024-09-21T14:21:41.910319+0000","last_clean":"2024-09-21T14:21:41.910319+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:41.910319+0000","last_undegraded":"2024-09-21T14:21:41.910319+0000","last_fullsized":"2024-09-21T14:21:41.910319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":4521871,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":1650,"num_read_kb":4222,"num_write":148,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.347434+0000","last_change":"2024-09-21T14:21:11.011596+0000","last_active":"2024-09-21T14:21:41.347434+0000","last_peered":"2024-09-21T14:21:41.347434+0000","last_clean":"2024-09-21T14:21:41.347434+0000","last_became_active":"2024-09-21T14:21:08.709180+0000","last_became_peered":"2024-09-21T14:21:08.709180+0000","last_unstale":"2024-09-21T14:21:41.347434+0000","last_undegraded":"2024-09-21T14:21:41.347434+0000","last_fullsized":"2024-09-21T14:21:41.347434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:00.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024667800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"40'103","reported_seq":1376,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.962911+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:41.962911+0000","last_peered":"2024-09-21T14:21:41.962911+0000","last_clean":"2024-09-21T14:21:41.962911+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:41.962911+0000","last_undegraded":"2024-09-21T14:21:41.962911+0000","last_fullsized":"2024-09-21T14:21:41.962911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":16853319,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1267,"num_read_kb":17940,"num_write":90,"num_write_kb":1766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.284737+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:41.284737+0000","last_peered":"2024-09-21T14:21:41.284737+0000","last_clean":"2024-09-21T14:21:41.284737+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:41.284737+0000","last_undegraded":"2024-09-21T14:21:41.284737+0000","last_fullsized":"2024-09-21T14:21:41.284737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"40'90","reported_seq":3367,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.287793+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:42.287793+0000","last_peered":"2024-09-21T14:21:42.287793+0000","last_clean":"2024-09-21T14:21:42.287793+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:42.287793+0000","last_undegraded":"2024-09-21T14:21:42.287793+0000","last_fullsized":"2024-09-21T14:21:42.287793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":9314384,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":3301,"num_read_kb":24532,"num_write":80,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"40'58","reported_seq":571,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.836763+0000","last_change":"2024-09-21T14:21:20.755906+0000","last_active":"2024-09-21T14:21:41.836763+0000","last_peered":"2024-09-21T14:21:41.836763+0000","last_clean":"2024-09-21T14:21:41.836763+0000","last_became_active":"2024-09-21T14:21:18.151157+0000","last_became_peered":"2024-09-21T14:21:18.151157+0000","last_unstale":"2024-09-21T14:21:41.836763+0000","last_undegraded":"2024-09-21T14:21:41.836763+0000","last_fullsized":"2024-09-21T14:21:41.836763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:25.967224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":7669218,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":497,"num_read_kb":14025,"num_write":64,"num_write_kb":3859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279575+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:41.279575+0000","last_peered":"2024-09-21T14:21:41.279575+0000","last_clean":"2024-09-21T14:21:41.279575+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:41.279575+0000","last_undegraded":"2024-09-21T14:21:41.279575+0000","last_fullsized":"2024-09-21T14:21:41.279575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":41,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.281723+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:41.281723+0000","last_peered":"2024-09-21T14:21:41.281723+0000","last_clean":"2024-09-21T14:21:41.281723+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:41.281723+0000","last_undegraded":"2024-09-21T14:21:41.281723+0000","last_fullsized":"2024-09-21T14:21:41.281723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"40'391","reported_seq":496,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.289294+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:42.289294+0000","last_peered":"2024-09-21T14:21:42.289294+0000","last_clean":"2024-09-21T14:21:42.289294+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:42.289294+0000","last_undegraded":"2024-09-21T14:21:42.289294+0000","last_fullsized":"2024-09-21T14:21:42.289294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":2071353,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":1667,"num_read_kb":1584,"num_write":647,"num_write_kb":1142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":42,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.310690+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:41.310690+0000","last_peered":"2024-09-21T14:21:41.310690+0000","last_clean":"2024-09-21T14:21:41.310690+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:41.310690+0000","last_undegraded":"2024-09-21T14:21:41.310690+0000","last_fullsized":"2024-09-21T14:21:41.310690+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'115","reported_seq":395,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.826338+0000","last_change":"2024-09-21T14:21:20.151933+0000","last_active":"2024-09-21T14:21:41.826338+0000","last_peered":"2024-09-21T14:21:41.826338+0000","last_clean":"2024-09-21T14:21:41.826338+0000","last_became_active":"2024-09-21T14:21:18.147902+0000","last_became_peered":"2024-09-21T14:21:18.147902+0000","last_unstale":"2024-09-21T14:21:41.826338+0000","last_undegraded":"2024-09-21T14:21:41.826338+0000","last_fullsized":"2024-09-21T14:21:41.826338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:09.190450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":3446646,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":365,"num_read_kb":2128,"num_write":122,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.279841+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:41.279841+0000","last_peered":"2024-09-21T14:21:41.279841+0000","last_clean":"2024-09-21T14:21:41.279841+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:41.279841+0000","last_undegraded":"2024-09-21T14:21:41.279841+0000","last_fullsized":"2024-09-21T14:21:41.279841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"41'260","reported_seq":1081,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.427869+0000","last_change":"2024-09-21T14:21:20.145954+0000","last_active":"2024-09-21T14:21:42.427869+0000","last_peered":"2024-09-21T14:21:42.427869+0000","last_clean":"2024-09-21T14:21:42.427869+0000","last_became_active":"2024-09-21T14:21:18.152479+0000","last_became_peered":"2024-09-21T14:21:18.152479+0000","last_unstale":"2024-09-21T14:21:42.427869+0000","last_undegraded":"2024-09-21T14:21:42.427869+0000","last_fullsized":"2024-09-21T14:21:42.427869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:27.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":19673601,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":1613,"num_read_kb":2798,"num_write":366,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.275098+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:41.275098+0000","last_peered":"2024-09-21T14:21:41.275098+0000","last_clean":"2024-09-21T14:21:41.275098+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:41.275098+0000","last_undegraded":"2024-09-21T14:21:41.275098+0000","last_fullsized":"2024-09-21T14:21:41.275098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"40'345","reported_seq":1582,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.962912+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:41.962912+0000","last_peered":"2024-09-21T14:21:41.962912+0000","last_clean":"2024-09-21T14:21:41.962912+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:41.962912+0000","last_undegraded":"2024-09-21T14:21:41.962912+0000","last_fullsized":"2024-09-21T14:21:41.962912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":8251443,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":3378,"num_read_kb":7241,"num_write":442,"num_write_kb":2178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"41'268","reported_seq":1769,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.438276+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:42.438276+0000","last_peered":"2024-09-21T14:21:42.438276+0000","last_clean":"2024-09-21T14:21:42.438276+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:42.438276+0000","last_undegraded":"2024-09-21T14:21:42.438276+0000","last_fullsized":"2024-09-21T14:21:42.438276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":57716938,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":3056,"num_read_kb":10847,"num_write":268,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"41'166","reported_seq":293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.346629+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:42.346629+0000","last_peered":"2024-09-21T14:21:42.346629+0000","last_clean":"2024-09-21T14:21:42.346629+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:42.346629+0000","last_undegraded":"2024-09-21T14:21:42.346629+0000","last_fullsized":"2024-09-21T14:21:42.346629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":3207626,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":336,"num_read_kb":1147,"num_write":178,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"40'91","reported_seq":750,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.824589+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:41.824589+0000","last_peered":"2024-09-21T14:21:41.824589+0000","last_clean":"2024-09-21T14:21:41.824589+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:41.824589+0000","last_undegraded":"2024-09-21T14:21:41.824589+0000","last_fullsized":"2024-09-21T14:21:41.824589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":14237900,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":667,"num_read_kb":1613,"num_write":179,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"41'199","reported_seq":332,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.431630+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:42.431630+0000","last_peered":"2024-09-21T14:21:42.431630+0000","last_clean":"2024-09-21T14:21:42.431630+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:42.431630+0000","last_undegraded":"2024-09-21T14:21:42.431630+0000","last_fullsized":"2024-09-21T14:21:42.431630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":6138736,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":664,"num_read_kb":833,"num_write":254,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"40'119","reported_seq":1443,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979690+0000","last_change":"2024-09-21T14:21:34.229265+0000","last_active":"2024-09-21T14:21:41.979690+0000","last_peered":"2024-09-21T14:21:41.979690+0000","last_clean":"2024-09-21T14:21:41.979690+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:21:41.979690+0000","last_undegraded":"2024-09-21T14:21:41.979690+0000","last_fullsized":"2024-09-21T14:21:41.979690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:04.756321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":15783528,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":1088,"num_read_kb":7557,"num_write":115,"num_write_kb":2819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"40'602","reported_seq":1538,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.999967+0000","last_change":"2024-09-21T14:21:20.756021+0000","last_active":"2024-09-21T14:21:41.999967+0000","last_peered":"2024-09-21T14:21:41.999967+0000","last_clean":"2024-09-21T14:21:41.999967+0000","last_became_active":"2024-09-21T14:21:18.152859+0000","last_became_peered":"2024-09-21T14:21:18.152859+0000","last_unstale":"2024-09-21T14:21:41.999967+0000","last_undegraded":"2024-09-21T14:21:41.999967+0000","last_fullsized":"2024-09-21T14:21:41.999967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.438917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":9891736,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":3798,"num_read_kb":9783,"num_write":1020,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"40'140","reported_seq":1815,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.840034+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:41.840034+0000","last_peered":"2024-09-21T14:21:41.840034+0000","last_clean":"2024-09-21T14:21:41.840034+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:41.840034+0000","last_undegraded":"2024-09-21T14:21:41.840034+0000","last_fullsized":"2024-09-21T14:21:41.840034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":16108775,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":1966,"num_read_kb":6775,"num_write":128,"num_write_kb":13529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"40'105","reported_seq":1138,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979380+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:41.979380+0000","last_peered":"2024-09-21T14:21:41.979380+0000","last_clean":"2024-09-21T14:21:41.979380+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:41.979380+0000","last_undegraded":"2024-09-21T14:21:41.979380+0000","last_fullsized":"2024-09-21T14:21:41.979380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":5807379,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":1350,"num_read_kb":7890,"num_write":94,"num_write_kb":2619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"40'134","reported_seq":820,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.818774+0000","last_change":"2024-09-21T14:21:20.160536+0000","last_active":"2024-09-21T14:21:41.818774+0000","last_peered":"2024-09-21T14:21:41.818774+0000","last_clean":"2024-09-21T14:21:41.818774+0000","last_became_active":"2024-09-21T14:21:18.151161+0000","last_became_peered":"2024-09-21T14:21:18.151161+0000","last_unstale":"2024-09-21T14:21:41.818774+0000","last_undegraded":"2024-09-21T14:21:41.818774+0000","last_fullsized":"2024-09-21T14:21:41.818774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:48.604891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":25245999,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":706,"num_read_kb":11107,"num_write":180,"num_write_kb":10542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'116","reported_seq":835,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:21:42.317868+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:42.317868+0000","last_peered":"2024-09-21T14:21:42.317868+0000","last_clean":"2024-09-21T14:21:42.317868+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:42.317868+0000","last_undegraded":"2024-09-21T14:21:42.317868+0000","last_fullsized":"2024-09-21T14:21:42.317868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":9966903,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1262,"num_read_kb":8120,"num_write":107,"num_write_kb":2495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"40'370","reported_seq":2621,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:21:41.979309+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:41.979309+0000","last_peered":"2024-09-21T14:21:41.979309+0000","last_clean":"2024-09-21T14:21:41.979309+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:41.979309+0000","last_undegraded":"2024-09-21T14:21:41.979309+0000","last_fullsized":"2024-09-21T14:21:41.979309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":13263935,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":4518,"num_read_kb":16246,"num_write":518,"num_write_kb":8109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":353378010,"num_objects":683,"num_object_clones":138,"num_object_copies":1366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":683,"num_whiteouts":322,"num_read":48865,"num_read_kb":252480,"num_write":6849,"num_write_kb":93112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12099584,"data_stored":99033782,"data_compressed":5342186,"data_compressed_allocated":6782976,"data_compressed_original":94882280,"omap_allocated":0,"internal_metadata":0},"log_size":5458,"ondisk_log_size":5458,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1245184,"data_stored":5534171,"data_compressed":270018,"data_compressed_allocated":372736,"data_compressed_original":5033984,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2011136,"data_stored":11900474,"data_compressed":637888,"data_compressed_allocated":876544,"data_compressed_original":11148439,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1536000,"data_stored":7938066,"data_compressed":366377,"data_compressed_allocated":516096,"data_compressed_original":7302802,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1986560,"data_stored":14353438,"data_compressed":708202,"data_compressed_allocated":790528,"data_compressed_original":13585730,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37912,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333928,"statfs":{"total":96636764160,"available":96597942272,"internally_reserved":0,"allocated":2768896,"data_stored":18924577,"data_compressed":1030028,"data_compressed_allocated":1363968,"data_compressed_original":17941368,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":2097152,"data_stored":13992528,"data_compressed":788059,"data_compressed_allocated":1081344,"data_compressed_original":13367323,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1728512,"data_stored":9245555,"data_compressed":549388,"data_compressed_allocated":659456,"data_compressed_original":8611335,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37800,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334040,"statfs":{"total":96636764160,"available":96598056960,"internally_reserved":0,"allocated":2641920,"data_stored":21337687,"data_compressed":1165481,"data_compressed_allocated":1355776,"data_compressed_original":20463587,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1978368,"data_stored":19988675,"data_compressed":1072829,"data_compressed_allocated":1228800,"data_compressed_original":19374051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1286144,"data_stored":9046425,"data_compressed":548811,"data_compressed_allocated":659456,"data_compressed_original":8599047,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1667072,"data_stored":13926989,"data_compressed":793900,"data_compressed_allocated":1093632,"data_compressed_original":13490203,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2260992,"data_stored":18142065,"data_compressed":997618,"data_compressed_allocated":1318912,"data_compressed_original":17367928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1552384,"data_stored":14232515,"data_compressed":711291,"data_compressed_allocated":794624,"data_compressed_original":13651266,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":999424,"data_stored":6646958,"data_compressed":309831,"data_compressed_allocated":438272,"data_compressed_original":6217362,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1560576,"data_stored":11708229,"data_compressed":637888,"data_compressed_allocated":876544,"data_compressed_original":11148439,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":794624,"data_stored":5341926,"data_compressed":270018,"data_compressed_allocated":372736,"data_compressed_original":5033984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:47.181 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:21:47.182 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:47.174+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:47.182 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:21:47.182 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:47.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-21T14:21:47.183 INFO:teuthology.orchestra.run.smithi073.stdout:2408 read 0x54ad80c thru 0x54b5e79 (0x866e bytes) 2024-09-21T14:21:47.186 INFO:teuthology.orchestra.run.smithi073.stdout:2409 writesame 0xea10289 thru 0xea17ea3 data_size 0x1(0x7c1b bytes) 2024-09-21T14:21:47.186 INFO:teuthology.orchestra.run.smithi073.stdout:2410 write 0x3d5b84e thru 0x3d64747 (0x8efa bytes) 2024-09-21T14:21:47.189 INFO:teuthology.orchestra.run.smithi073.stdout:2411 read 0x65af1c0 thru 0x65b840c (0x924d bytes) 2024-09-21T14:21:47.189 INFO:teuthology.orchestra.run.smithi073.stdout:2412 read 0x4aedb3c thru 0x4af8345 (0xa80a bytes) 2024-09-21T14:21:47.193 INFO:teuthology.orchestra.run.smithi073.stdout:2414 punch from 0x608590b to 0x60906f1, (0xade6 bytes) 2024-09-21T14:21:47.193 INFO:teuthology.orchestra.run.smithi073.stdout:2417 punch from 0x886f00 to 0x889ed0, (0x2fd0 bytes) 2024-09-21T14:21:47.193 INFO:teuthology.orchestra.run.smithi073.stdout:2419 punch from 0xc4a83a0 to 0xc4b5e69, (0xdac9 bytes) 2024-09-21T14:21:47.193 INFO:teuthology.orchestra.run.smithi073.stdout:2420 read 0x1ba44bc thru 0x1badd45 (0x988a bytes) 2024-09-21T14:21:47.193 INFO:teuthology.orchestra.run.smithi073.stdout:2421 write 0x9404af7 thru 0x94077d8 (0x2ce2 bytes) 2024-09-21T14:21:47.200 INFO:teuthology.orchestra.run.smithi073.stdout:2422 write 0x5fce51b thru 0x5fdc553 (0xe039 bytes) 2024-09-21T14:21:47.211 INFO:teuthology.orchestra.run.smithi073.stdout:2424 write 0x3f6502e thru 0x3f6732e (0x2301 bytes) 2024-09-21T14:21:47.223 INFO:teuthology.orchestra.run.smithi073.stdout:2425 write 0xad7cf9f thru 0xad82ff1 (0x6053 bytes) 2024-09-21T14:21:47.232 INFO:teuthology.orchestra.run.smithi073.stdout:2426 read 0x8fc5107 thru 0x8fcbeff (0x6df9 bytes) 2024-09-21T14:21:47.233 INFO:teuthology.orchestra.run.smithi073.stdout:2427 read 0xa0b4f56 thru 0xa0bcaca (0x7b75 bytes) 2024-09-21T14:21:47.233 INFO:teuthology.orchestra.run.smithi073.stdout:2428 write 0x2ee369c thru 0x2eefa68 (0xc3cd bytes) 2024-09-21T14:21:47.235 INFO:teuthology.orchestra.run.smithi073.stdout:2431 write 0xb73ea44 thru 0xb747aa9 (0x9066 bytes) 2024-09-21T14:21:47.249 INFO:teuthology.orchestra.run.smithi073.stdout:2432 read 0x19ef272 thru 0x19fc5fc (0xd38b bytes) 2024-09-21T14:21:47.249 INFO:teuthology.orchestra.run.smithi073.stdout:2433 trunc from 0xea17ea4 to 0xdfab5f1 2024-09-21T14:21:47.255 INFO:teuthology.orchestra.run.smithi073.stdout:2434 writesame 0x76600de thru 0x7668388 data_size 0x1(0x82ab bytes) 2024-09-21T14:21:47.270 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:47.270 INFO:teuthology.orchestra.run.smithi073.stdout:2436 read 0x15f1427 thru 0x15f7bc1 (0x679b bytes) 2024-09-21T14:21:47.270 INFO:teuthology.orchestra.run.smithi073.stdout:2437 read 0x9dfe9 thru 0xad473 (0xf48b bytes) 2024-09-21T14:21:47.272 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:47.272 INFO:teuthology.orchestra.run.smithi073.stdout:2439 read 0x44f2570 thru 0x44f2bfb (0x68c bytes) 2024-09-21T14:21:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:2440 writesame 0xaab8b51 thru 0xaac70fb data_size 0xb(0xe5ab bytes) 2024-09-21T14:21:47.275 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:47.275+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:47.290 INFO:teuthology.orchestra.run.smithi073.stdout:2441 trunc from 0xdfab5f1 to 0x87a2220 2024-09-21T14:21:47.314 INFO:teuthology.orchestra.run.smithi073.stdout:2442 trunc from 0x87a2220 to 0x1c73a1b 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42565 3.5 3.8e5460e5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.317636+0000", 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0018337990000000001, 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001858976, 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "op must wait for map", 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.319 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42565 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317636+0000", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317636+0000", 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317634+0000", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317642+0000", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4849999999999992e-06 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317660+0000", 2024-09-21T14:21:47.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7944999999999998e-05 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317669+0000", 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317721+0000", 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1119000000000003e-05 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.317727+0000", 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4529999999999999e-06 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:47.323 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:21:47.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:47.357 INFO:teuthology.orchestra.run.smithi073.stdout:2443 writesame 0x8de2bf thru 0x8e7478 data_size 0x17(0x91ba bytes) 2024-09-21T14:21:47.365 INFO:teuthology.orchestra.run.smithi073.stdout:2445 trunc from 0x1c73a1b to 0x4e54e8f 2024-09-21T14:21:47.371 INFO:teuthology.orchestra.run.smithi073.stdout:2446 flatten 2024-09-21T14:21:47.376 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:47.376+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:47.476 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:47.476+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:47.495 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:47.495 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:47.496 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:47.496 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:47.496 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:47.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:47.563 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:21:47.563+0000 7f0ce787f640 -1 mon.a@0(leader).osd e46 definitely_dead 0 2024-09-21T14:21:47.577 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:47.577+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:47.650 INFO:teuthology.orchestra.run.smithi073.stdout:2447 trunc from 0x4e54e8f to 0x2eeb12d 2024-09-21T14:21:47.715 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.258841844999999, 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.258830944, 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:21:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.258820536000002, 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:21:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.258812594999998, 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:21:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.182074610000001, 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:21:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:21:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.182056200000002, 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:21:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.182030450999999, 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:21:47.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.182009422, 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:21:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.751 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181983572, 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:21:47.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.755 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181965092999999, 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.756 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:21:47.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:21:47.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:47.760 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181946970999999, 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:47.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:21:47.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181926609000001, 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.765 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:21:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:21:47.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181906916999999, 2024-09-21T14:21:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:21:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181889939000001, 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:21:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:21:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:21:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181870267000001, 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:21:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181840804, 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:21:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181798558000001, 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:21:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:21:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.18177957, 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:21:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:21:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181756212, 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:21:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.181732859, 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:47.804 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:47.805 INFO:teuthology.orchestra.run.smithi073.stdout:2450 read 0x3d0972 thru 0x3dfb78 (0xf207 bytes) 2024-09-21T14:21:47.805 INFO:teuthology.orchestra.run.smithi073.stdout:2451 read 0x57c164 thru 0x580913 (0x47b0 bytes) 2024-09-21T14:21:47.805 INFO:teuthology.orchestra.run.smithi073.stdout:2452 clone 19 order 22 su 2097152 sc 13 2024-09-21T14:21:47.806 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:47.807 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:47.716+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:47.847 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:47.847+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:47.917 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:47.917+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:47.950 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:47.950 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:47.950 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:21:47.950 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:47.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:48.043 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.043+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:48.101 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:48.101 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:48.101 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:48.101 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:48.102 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:48.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:48.151 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:48.151+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:48.219 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.219+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:48.254 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 38.712314454000001, 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:48.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:48.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:48.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43094 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86678828900000005, 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355065000000002, 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43094 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387470+0000", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5480000000000003e-06 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387475+0000", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1560000000000003e-06 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387479+0000", 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7330000000000001e-06 2024-09-21T14:21:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427814+0000", 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040335226000000002 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427821+0000", 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6889999999999998e-06 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43095 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86677151399999997, 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040366234000000001, 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43095 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387484+0000", 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.857e-06 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387489+0000", 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387493+0000", 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2830000000000003e-06 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427842+0000", 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040348342000000002 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427849+0000", 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9759999999999998e-06 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86675696099999999, 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.270 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.273 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86673809199999996, 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:21:48.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86672406000000002, 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.278 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:21:48.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86670806099999997, 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:48.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86669316500000004, 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86667822900000002, 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:48.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86666324900000002, 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:48.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:21:48.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86664863299999995, 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.298 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:48.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:21:48.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86663358599999996, 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.302 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86661902700000004, 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.306 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:21:48.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86660430200000005, 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.310 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86658986199999999, 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:21:48.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86657551200000005, 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.318 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:21:48.321 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86655544600000001, 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.322 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:21:48.325 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86654053600000003, 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.326 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:48.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:21:48.329 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86652559600000001, 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.330 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:48.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:21:48.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.86650590299999997, 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:48.334 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:48.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:48.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:48.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:48.338 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:48.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:48.340 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.2. 2024-09-21T14:21:48.341 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:48.339+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:48.440 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:48.440+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:48.445 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:48.446 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:48.454 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:48.541 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.542+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:48.596 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:48.596 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:48.604 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:48.642 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.642+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:48.707 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:48.707 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:48.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43741 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3166201179999999, 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036952564, 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43741 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390192+0000", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.14e-06 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390199+0000", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5150000000000003e-06 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390204+0000", 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000001e-06 2024-09-21T14:21:48.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.426948+0000", 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036744128000000001 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427034+0000", 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6402000000000004e-05 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427137+0000", 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010275799999999999 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43742 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3165953109999999, 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037087785999999998, 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43742 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390211+0000", 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1019999999999999e-06 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390216+0000", 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2870000000000004e-06 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390221+0000", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427168+0000", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036947520999999997 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427213+0000", 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4718000000000001e-05 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427297+0000", 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3939999999999999e-05 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43743 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3165797590000001, 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037293508000000003, 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43743 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390224+0000", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390227+0000", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390232+0000", 2024-09-21T14:21:48.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5799999999999999e-06 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390237+0000", 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7890000000000002e-06 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427333+0000", 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037095876999999999 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6895000000000001e-05 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427518+0000", 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8611999999999997e-05 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43744 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3165635689999999, 2024-09-21T14:21:48.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037409745000000001, 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43744 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390243+0000", 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:21:48.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390249+0000", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390253+0000", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7829999999999998e-06 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427550+0000", 2024-09-21T14:21:48.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037296636000000001 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427608+0000", 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8100000000000003e-05 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427651+0000", 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2528999999999999e-05 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43745 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316547259, 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037596321000000002, 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43745 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390259+0000", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0770000000000002e-06 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390264+0000", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3430000000000003e-06 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390268+0000", 2024-09-21T14:21:48.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0470000000000004e-06 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427691+0000", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037422596000000002 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427784+0000", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2780000000000003e-05 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427854+0000", 2024-09-21T14:21:48.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9770000000000005e-05 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43746 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3165333640000001, 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037742629, 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.729 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43746 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390273+0000", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1150000000000001e-06 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390279+0000", 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6799999999999998e-06 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390283+0000", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427891+0000", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037607640999999997 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427973+0000", 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2170999999999996e-05 2024-09-21T14:21:48.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428014+0000", 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0806000000000002e-05 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43747 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316484974, 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037837335999999999, 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43747 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390319+0000", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390323+0000", 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390332+0000", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9600000000000006e-06 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390339+0000", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8959999999999997e-06 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428048+0000", 2024-09-21T14:21:48.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037709135999999997 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428117+0000", 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8534999999999993e-05 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428157+0000", 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0117000000000002e-05 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43748 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3164601970000001, 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037945300000000001, 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43748 2024-09-21T14:21:48.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390346+0000", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.464e-06 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390354+0000", 2024-09-21T14:21:48.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9619999999999993e-06 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390359+0000", 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428184+0000", 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037824759999999999 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428230+0000", 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6112000000000002e-05 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428290+0000", 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9327000000000003e-05 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43749 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.742 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316440501, 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038038324999999998, 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43749 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:48.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390368+0000", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1579999999999998e-06 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390375+0000", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390380+0000", 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9609999999999996e-06 2024-09-21T14:21:48.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428319+0000", 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037939925999999999 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428364+0000", 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4623999999999998e-05 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428402+0000", 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43752 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316420712, 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038243006000000003, 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43752 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390383+0000", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390386+0000", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0640000000000001e-06 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390392+0000", 2024-09-21T14:21:48.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4269999999999996e-06 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390397+0000", 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7650000000000001e-06 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428433+0000", 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038036164999999997 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428525+0000", 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1831999999999998e-05 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428627+0000", 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102099 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43753 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.751 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3164034689999999, 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038351382000000003, 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43753 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:48.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390403+0000", 2024-09-21T14:21:48.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0990000000000001e-06 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390410+0000", 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7299999999999999e-06 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390415+0000", 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0730000000000004e-06 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428653+0000", 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038237924 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428697+0000", 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4367999999999999e-05 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428752+0000", 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5535000000000003e-05 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43791 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316385285, 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038564714, 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43791 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390421+0000", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.345e-06 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390428+0000", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6669999999999999e-06 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390433+0000", 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.169e-06 2024-09-21T14:21:48.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428791+0000", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038358033 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428890+0000", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9216999999999996e-05 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428984+0000", 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3666999999999996e-05 2024-09-21T14:21:48.759 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43792 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3163671189999999, 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038678535, 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43792 2024-09-21T14:21:48.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:48.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390444+0000", 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390451+0000", 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390460+0000", 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9150000000000008e-06 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429008+0000", 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038547887000000003 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429052+0000", 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3509e-05 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429116+0000", 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4078999999999994e-05 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43802 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3163386100000001, 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038886423000000003, 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.764 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43802 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390468+0000", 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.328e-06 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390476+0000", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1939999999999998e-06 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390481+0000", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2449999999999998e-06 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429158+0000", 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038676675000000001 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429249+0000", 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0657999999999998e-05 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429352+0000", 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103729 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43803 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316318753, 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039086681999999998, 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43803 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:48.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390485+0000", 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390488+0000", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1789999999999998e-06 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390494+0000", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3040000000000001e-06 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390499+0000", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429393+0000", 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038894038999999998 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429487+0000", 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4504000000000002e-05 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429572+0000", 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.509e-05 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.772 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43819 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3163014159999999, 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039269071000000003, 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43819 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.773 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390505+0000", 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1490000000000001e-06 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390511+0000", 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0290000000000002e-06 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390516+0000", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8060000000000002e-06 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429608+0000", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039092162 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429697+0000", 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8712e-05 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429772+0000", 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5586000000000004e-05 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43820 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316058664, 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039230445000000003, 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.777 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43820 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390745+0000", 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390749+0000", 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.799e-06 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390770+0000", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1035000000000001e-05 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390777+0000", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9890000000000003e-06 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039033209999999999 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429904+0000", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4094999999999999e-05 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429976+0000", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1931e-05 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43832 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316021836, 2024-09-21T14:21:48.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039328466999999999, 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43832 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390782+0000", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390785+0000", 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8870000000000001e-06 2024-09-21T14:21:48.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390792+0000", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.951e-06 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390797+0000", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1780000000000002e-06 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430003+0000", 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039205885000000003 2024-09-21T14:21:48.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430046+0000", 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2867000000000001e-05 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430111+0000", 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5242999999999999e-05 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.785 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43833 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.316002874, 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039492185999999999, 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43833 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390801+0000", 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390804+0000", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390810+0000", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4999999999999996e-06 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390815+0000", 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430137+0000", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039321964000000001 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430192+0000", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4787000000000002e-05 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430294+0000", 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102186 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43845 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 151552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.3159851520000001, 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039630430000000001, 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43845 2024-09-21T14:21:48.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:48.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390821+0000", 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2689999999999998e-06 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390833+0000", 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2248e-05 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:48.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390838+0000", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9540000000000003e-06 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430335+0000", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039496297999999999 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430389+0000", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4293999999999999e-05 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:48.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430450+0000", 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0757999999999997e-05 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:48.794 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:48.795 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:48.796 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:48.795+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:48.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:48.898 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:48.898 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:48.898 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:48.898 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:48.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:48.931 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.931+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:48.997 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:48.997+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:49.005 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:49.005 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:49.009 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:49.009 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:49.009 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:49.010 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:49.010 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:49.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:49.021 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:49.021 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:49.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:49.036 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:49.052 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:49.052 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:49.068 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:49.068 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:49.084 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:49.084 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:49.097 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:49.098+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:49.100 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:49.101 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:49.116 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:49.116 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:49.122 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.122 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:49.198 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:49.199+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:49.233 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.233 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:49.300 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:49.300+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:49.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:21:49.350 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.350 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.360 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:49.401 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:49.401+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:49.464 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.465 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:49.518 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:49.518+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:49.576 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.576 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:49.602 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:49.602+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:49.703 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:49.703+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:49.727 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.727 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:49.804 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:49.804+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:49.837 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.837 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:49.905 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:49.905+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:49.947 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:49.947 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:49.956 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:50.006 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:50.006+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:50.057 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.058 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:50.107 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:50.107+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:50.208 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:50.208+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:50.209 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.209 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:50.310 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:50.310+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:50.332 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:21:50.378 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.378 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:50.410 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:50.410+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:50.483 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T14:21:50.491 INFO:teuthology.orchestra.run.smithi073.stdout:2457 trunc from 0x2eeb12d to 0x1a46dd0 2024-09-21T14:21:50.512 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:50.512+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:50.524 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.524 INFO:teuthology.orchestra.run.smithi073.stdout:2460 trunc from 0x1a46dd0 to 0x215f14a 2024-09-21T14:21:50.549 INFO:teuthology.orchestra.run.smithi073.stdout:2461 write 0xd1ec290 thru 0xd1ed826 (0x1597 bytes) 2024-09-21T14:21:50.552 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.552 INFO:teuthology.orchestra.run.smithi073.stdout:2466 read 0x374fb75 thru 0x3758cca (0x9156 bytes) 2024-09-21T14:21:50.552 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.558 INFO:teuthology.orchestra.run.smithi073.stdout:2468 trunc from 0xd1ed827 to 0x7c8939f 2024-09-21T14:21:50.571 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.571 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.573 INFO:teuthology.orchestra.run.smithi073.stdout:2470 punch from 0x2354528 to 0x2364197, (0xfc6f bytes) 2024-09-21T14:21:50.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:50.587 INFO:teuthology.orchestra.run.smithi073.stdout:2471 writesame 0xbf84d20 thru 0xbf91f9b data_size 0x26(0xd27c bytes) 2024-09-21T14:21:50.587 INFO:teuthology.orchestra.run.smithi073.stdout:2472 punch from 0x4c0c45b to 0x4c0c6e7, (0x28c bytes) 2024-09-21T14:21:50.587 INFO:teuthology.orchestra.run.smithi073.stdout:2473 read 0x2f95e0d thru 0x2f9ae92 (0x5086 bytes) 2024-09-21T14:21:50.588 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.591 INFO:teuthology.orchestra.run.smithi073.stdout:2476 trunc from 0xbf91f9c to 0x947e768 2024-09-21T14:21:50.596 INFO:teuthology.orchestra.run.smithi073.stdout:2479 read 0x79904f2 thru 0x79945fc (0x410b bytes) 2024-09-21T14:21:50.603 INFO:teuthology.orchestra.run.smithi073.stdout:2480 trunc from 0x947e768 to 0xe522b84 2024-09-21T14:21:50.610 INFO:teuthology.orchestra.run.smithi073.stdout:2483 read 0x18d4caa thru 0x18d9794 (0x4aeb bytes) 2024-09-21T14:21:50.610 INFO:teuthology.orchestra.run.smithi073.stdout:2484 read 0xb44946d thru 0xb4517eb (0x837f bytes) 2024-09-21T14:21:50.611 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.611 INFO:teuthology.orchestra.run.smithi073.stdout:2486 read 0x46e929b thru 0x46eaafc (0x1862 bytes) 2024-09-21T14:21:50.612 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:50.612+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:50.616 INFO:teuthology.orchestra.run.smithi073.stdout:2487 trunc from 0xe522b84 to 0x78f72d9 2024-09-21T14:21:50.621 INFO:teuthology.orchestra.run.smithi073.stdout:2488 write 0x6ff9dab thru 0x6ffef77 (0x51cd bytes) 2024-09-21T14:21:50.624 INFO:teuthology.orchestra.run.smithi073.stdout:2489 read 0x72fafa9 thru 0x73034e3 (0x853b bytes) 2024-09-21T14:21:50.624 INFO:teuthology.orchestra.run.smithi073.stdout:2490 read 0x4a625b4 thru 0x4a71f96 (0xf9e3 bytes) 2024-09-21T14:21:50.626 INFO:teuthology.orchestra.run.smithi073.stdout:2492 trunc from 0x78f72d9 to 0x8ac26b0 2024-09-21T14:21:50.632 INFO:teuthology.orchestra.run.smithi073.stdout:2493 read 0x3c10321 thru 0x3c15b59 (0x5839 bytes) 2024-09-21T14:21:50.633 INFO:teuthology.orchestra.run.smithi073.stdout:2494 read 0x4b08cba thru 0x4b16ec2 (0xe209 bytes) 2024-09-21T14:21:50.633 INFO:teuthology.orchestra.run.smithi073.stdout:2496 writesame 0x29f18f2 thru 0x29ff365 data_size 0x7c(0xda74 bytes) 2024-09-21T14:21:50.634 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.634 INFO:teuthology.orchestra.run.smithi073.stdout:2500 read 0x41c565a thru 0x41d0747 (0xb0ee bytes) 2024-09-21T14:21:50.640 INFO:teuthology.orchestra.run.smithi073.stdout:2501 trunc from 0x8ac26b0 to 0xdb96d4f 2024-09-21T14:21:50.655 INFO:teuthology.orchestra.run.smithi073.stdout:2503 trunc from 0xdb96d4f to 0x5f2095b 2024-09-21T14:21:50.675 INFO:teuthology.orchestra.run.smithi073.stdout:2505 writesame 0xbb3393b thru 0xbb3b060 data_size 0x2(0x7726 bytes) 2024-09-21T14:21:50.675 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.676 INFO:teuthology.orchestra.run.smithi073.stdout:2507 write 0x15fa9d9 thru 0x15fc8b2 (0x1eda bytes) 2024-09-21T14:21:50.677 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:50.677 INFO:teuthology.orchestra.run.smithi073.stdout:2513 write 0x80c1cc2 thru 0x80c4b04 (0x2e43 bytes) 2024-09-21T14:21:50.680 INFO:teuthology.orchestra.run.smithi073.stdout:2514 write 0x5b9804f thru 0x5b9f151 (0x7103 bytes) 2024-09-21T14:21:50.683 INFO:teuthology.orchestra.run.smithi073.stdout:2515 read 0x881730d thru 0x881dc62 (0x6956 bytes) 2024-09-21T14:21:50.684 INFO:teuthology.orchestra.run.smithi073.stdout:2516 clone 20 order 24 su 65536 sc 10 2024-09-21T14:21:50.709 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.709 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.713 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:50.714+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:50.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:50.814 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:50.815+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:50.859 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.859 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.868 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:50.915 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:50.915+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:50.970 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:50.970 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:50.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:51.016 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:51.016+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:51.117 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:51.117+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:51.121 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:51.122 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:51.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:51.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:51.217 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:51.217+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:51.237 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:51.238 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:51.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:51.252 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:51.253 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:51.269 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:51.269 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:51.285 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:51.285 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:51.301 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:51.301 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:51.318 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:51.318 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:51.318 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:51.318+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:51.334 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:51.334 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:51.352 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:51.352 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:51.356 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:51.356 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:51.356 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:51.356 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:51.357 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:51.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:51.369 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:51.369 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:51.419 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:51.420+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:51.473 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.931308596999997, 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:21:51.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:21:51.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:21:51.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:21:51.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43094 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0857824320000002, 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355065000000002, 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43094 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387470+0000", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5480000000000003e-06 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387475+0000", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1560000000000003e-06 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387479+0000", 2024-09-21T14:21:51.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7330000000000001e-06 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427814+0000", 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040335226000000002 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427821+0000", 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6889999999999998e-06 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43095 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0857656569999996, 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040366234000000001, 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43095 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387484+0000", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.857e-06 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387489+0000", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387493+0000", 2024-09-21T14:21:51.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2830000000000003e-06 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427842+0000", 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040348342000000002 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427849+0000", 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9759999999999998e-06 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0857511039999999, 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:21:51.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.085732235, 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.496 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0857182029999999, 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:21:51.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.500 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0857022040000004, 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:21:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.504 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856873079999998, 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:21:51.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856723720000003, 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.509 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:21:51.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.512 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856573919999999, 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:21:51.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:21:51.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856427760000003, 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.517 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:51.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856277289999996, 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.521 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0856131700000002, 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.525 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:21:51.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:21:51.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855984449999996, 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.529 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:21:51.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855840050000003, 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:21:51.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:21:51.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855696549999996, 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:21:51.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:21:51.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.541 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855495890000002, 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:21:51.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855346790000002, 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:21:51.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855197390000004, 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:21:51.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:21:51.553 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0855000459999999, 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:51.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:21:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:51.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:51.558 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:51.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:51.560 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:51.560+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:51.660 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:51.661+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:51.667 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:51.668 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:51.676 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:51.761 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:51.762+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:51.778 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:51.778 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:51.778 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:51.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:51.871 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:51.871+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:51.930 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:51.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43741 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5396708080000003, 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036952564, 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43741 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.931 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:21:51.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390192+0000", 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.14e-06 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390199+0000", 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5150000000000003e-06 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390204+0000", 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000001e-06 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.426948+0000", 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036744128000000001 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427034+0000", 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6402000000000004e-05 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427137+0000", 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010275799999999999 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43742 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5396460010000004, 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037087785999999998, 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.935 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.936 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43742 2024-09-21T14:21:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390211+0000", 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1019999999999999e-06 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390216+0000", 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2870000000000004e-06 2024-09-21T14:21:51.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390221+0000", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427168+0000", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036947520999999997 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427213+0000", 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4718000000000001e-05 2024-09-21T14:21:51.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427297+0000", 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3939999999999999e-05 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43743 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5396304489999997, 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037293508000000003, 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.941 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43743 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:21:51.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390224+0000", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390227+0000", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390232+0000", 2024-09-21T14:21:51.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5799999999999999e-06 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390237+0000", 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7890000000000002e-06 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427333+0000", 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037095876999999999 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6895000000000001e-05 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427518+0000", 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8611999999999997e-05 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43744 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5396142590000004, 2024-09-21T14:21:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037409745000000001, 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43744 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390243+0000", 2024-09-21T14:21:51.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390249+0000", 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390253+0000", 2024-09-21T14:21:51.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7829999999999998e-06 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427550+0000", 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037296636000000001 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427608+0000", 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8100000000000003e-05 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427651+0000", 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2528999999999999e-05 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43745 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.539597949, 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037596321000000002, 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.950 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43745 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390259+0000", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0770000000000002e-06 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390264+0000", 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3430000000000003e-06 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390268+0000", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0470000000000004e-06 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427691+0000", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037422596000000002 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427784+0000", 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2780000000000003e-05 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427854+0000", 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9770000000000005e-05 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43746 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5395840539999996, 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037742629, 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.954 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43746 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:51.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390273+0000", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1150000000000001e-06 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390279+0000", 2024-09-21T14:21:51.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6799999999999998e-06 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390283+0000", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427891+0000", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037607640999999997 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427973+0000", 2024-09-21T14:21:51.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2170999999999996e-05 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428014+0000", 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0806000000000002e-05 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43747 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.958 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5395356639999997, 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037837335999999999, 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43747 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:51.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390319+0000", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390323+0000", 2024-09-21T14:21:51.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390332+0000", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9600000000000006e-06 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390339+0000", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8959999999999997e-06 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428048+0000", 2024-09-21T14:21:51.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037709135999999997 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428117+0000", 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8534999999999993e-05 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428157+0000", 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0117000000000002e-05 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.962 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43748 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5395108870000005, 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037945300000000001, 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43748 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.963 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390346+0000", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.464e-06 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390354+0000", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9619999999999993e-06 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390359+0000", 2024-09-21T14:21:51.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428184+0000", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037824759999999999 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428230+0000", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6112000000000002e-05 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428290+0000", 2024-09-21T14:21:51.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9327000000000003e-05 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43749 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5394911909999998, 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038038324999999998, 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.967 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43749 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:21:51.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390368+0000", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1579999999999998e-06 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390375+0000", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390380+0000", 2024-09-21T14:21:51.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9609999999999996e-06 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428319+0000", 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037939925999999999 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428364+0000", 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4623999999999998e-05 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428402+0000", 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43752 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5394714020000002, 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038243006000000003, 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43752 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390383+0000", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390386+0000", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0640000000000001e-06 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390392+0000", 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4269999999999996e-06 2024-09-21T14:21:51.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390397+0000", 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7650000000000001e-06 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428433+0000", 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038036164999999997 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428525+0000", 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1831999999999998e-05 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428627+0000", 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102099 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43753 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:51.975 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5394541589999999, 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038351382000000003, 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43753 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390403+0000", 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0990000000000001e-06 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390410+0000", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7299999999999999e-06 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390415+0000", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0730000000000004e-06 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428653+0000", 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038237924 2024-09-21T14:21:51.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428697+0000", 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4367999999999999e-05 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428752+0000", 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5535000000000003e-05 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43791 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.539435975, 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038564714, 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43791 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:51.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390421+0000", 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.345e-06 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390428+0000", 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6669999999999999e-06 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390433+0000", 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.169e-06 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428791+0000", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038358033 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428890+0000", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9216999999999996e-05 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428984+0000", 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3666999999999996e-05 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.983 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43792 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5394178089999997, 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038678535, 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43792 2024-09-21T14:21:51.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390444+0000", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390451+0000", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390460+0000", 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9150000000000008e-06 2024-09-21T14:21:51.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429008+0000", 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038547887000000003 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429052+0000", 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3509e-05 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429116+0000", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4078999999999994e-05 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43802 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5393892999999998, 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038886423000000003, 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.988 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43802 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390468+0000", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.328e-06 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390476+0000", 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1939999999999998e-06 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390481+0000", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2449999999999998e-06 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429158+0000", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038676675000000001 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429249+0000", 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0657999999999998e-05 2024-09-21T14:21:51.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429352+0000", 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103729 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43803 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.539369443, 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039086681999999998, 2024-09-21T14:21:51.992 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43803 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390485+0000", 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390488+0000", 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1789999999999998e-06 2024-09-21T14:21:51.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390494+0000", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3040000000000001e-06 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390499+0000", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429393+0000", 2024-09-21T14:21:51.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038894038999999998 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429487+0000", 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4504000000000002e-05 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429572+0000", 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.509e-05 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:51.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43819 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5393521059999999, 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039269071000000003, 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43819 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.997 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:51.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390505+0000", 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1490000000000001e-06 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390511+0000", 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0290000000000002e-06 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:51.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390516+0000", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8060000000000002e-06 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429608+0000", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039092162 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429697+0000", 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8712e-05 2024-09-21T14:21:52.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429772+0000", 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5586000000000004e-05 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43820 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5391093539999998, 2024-09-21T14:21:52.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039230445000000003, 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43820 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390745+0000", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390749+0000", 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.799e-06 2024-09-21T14:21:52.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390770+0000", 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1035000000000001e-05 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390777+0000", 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9890000000000003e-06 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039033209999999999 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429904+0000", 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4094999999999999e-05 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429976+0000", 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1931e-05 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43832 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.539072526, 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039328466999999999, 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43832 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390782+0000", 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390785+0000", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8870000000000001e-06 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390792+0000", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.951e-06 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390797+0000", 2024-09-21T14:21:52.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1780000000000002e-06 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430003+0000", 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039205885000000003 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430046+0000", 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2867000000000001e-05 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430111+0000", 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5242999999999999e-05 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43833 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5390535639999996, 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039492185999999999, 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43833 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390801+0000", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390804+0000", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390810+0000", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4999999999999996e-06 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390815+0000", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430137+0000", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039321964000000001 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430192+0000", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4787000000000002e-05 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430294+0000", 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102186 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43845 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 151552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.5390358419999997, 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039630430000000001, 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.014 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43845 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390821+0000", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2689999999999998e-06 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390833+0000", 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2248e-05 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390838+0000", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9540000000000003e-06 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430335+0000", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039496297999999999 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430389+0000", 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4293999999999999e-05 2024-09-21T14:21:52.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430450+0000", 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0757999999999997e-05 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.018 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.020 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:52.020 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:52.020+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:52.120 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:52.121+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:52.168 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:52.168 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:52.169 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.174 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone19 intact 2024-09-21T14:21:52.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:52.222 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:52.222+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:52.280 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:52.280 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:52.280 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:52.301 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T14:21:52.333 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:52.334+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:52.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:21:52.364 INFO:teuthology.orchestra.run.smithi073.stdout:2517 read 0x32bf901 thru 0x32c1411 (0x1b11 bytes) 2024-09-21T14:21:52.373 INFO:teuthology.orchestra.run.smithi073.stdout:2518 trunc from 0xbb3b061 to 0x54049bb 2024-09-21T14:21:52.410 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699708373, 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:21:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:21:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699697062999999, 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699681687000002, 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699664159000001, 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:21:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:21:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699645208, 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699628828000002, 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:21:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699619599999998, 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:21:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:21:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:21:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699608204, 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:21:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699585476999999, 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699569221000001, 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:21:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699554465999999, 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:21:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:21:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699545411999999, 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699534367999998, 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:21:52.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:21:52.465 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699518966999999, 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.466 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:21:52.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.470 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699502269, 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:52.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:21:52.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:21:52.474 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699486661999998, 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:21:52.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699469539999999, 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:21:52.479 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:21:52.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699450929000001, 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.487 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699431599, 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.488 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:21:52.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:21:52.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.699411961999999, 2024-09-21T14:21:52.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:21:52.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.497 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.497 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:52.499 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:52.498+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:52.599 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:52.599+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:52.623 INFO:teuthology.orchestra.run.smithi073.stdout:2519 write 0xc7913ad thru 0xc79510f (0x3d63 bytes) 2024-09-21T14:21:52.626 INFO:teuthology.orchestra.run.smithi073.stdout:2520 writesame 0xa9a88dc thru 0xa9b18c7 data_size 0x4(0x8fec bytes) 2024-09-21T14:21:52.632 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:52.644 INFO:teuthology.orchestra.run.smithi073.stdout:2522 trunc from 0xc795110 to 0x1fefff5 2024-09-21T14:21:52.652 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:52.653 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:52.653 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.657 INFO:teuthology.orchestra.run.smithi073.stdout:2523 trunc from 0x1fefff5 to 0xf84087 2024-09-21T14:21:52.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:52.664 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:52.685 INFO:teuthology.orchestra.run.smithi073.stdout:2526 writesame 0xd847151 thru 0xd851c88 data_size 0x4(0xab38 bytes) 2024-09-21T14:21:52.699 INFO:teuthology.orchestra.run.smithi073.stdout:2527 trunc from 0xd851c89 to 0x22e514c 2024-09-21T14:21:52.705 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:52.705+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:52.718 INFO:teuthology.orchestra.run.smithi073.stdout:2528 writesame 0x4bd3f01 thru 0x4bda50c data_size 0x2a(0x660c bytes) 2024-09-21T14:21:52.732 INFO:teuthology.orchestra.run.smithi073.stdout:2530 write 0xb462b3f thru 0xb470c42 (0xe104 bytes) 2024-09-21T14:21:52.739 INFO:teuthology.orchestra.run.smithi073.stdout:2531 writesame 0x514c8e1 thru 0x514ecd3 data_size 0x1(0x23f3 bytes) 2024-09-21T14:21:52.751 INFO:teuthology.orchestra.run.smithi073.stdout:2532 write 0xcba14a9 thru 0xcba6915 (0x546d bytes) 2024-09-21T14:21:52.754 INFO:teuthology.orchestra.run.smithi073.stdout:2533 trunc from 0xcba6916 to 0xc38b190 2024-09-21T14:21:52.762 INFO:teuthology.orchestra.run.smithi073.stdout:2535 write 0xb8a76f0 thru 0xb8b6106 (0xea17 bytes) 2024-09-21T14:21:52.769 INFO:teuthology.orchestra.run.smithi073.stdout:2536 writesame 0x8bc1542 thru 0x8bd0c06 data_size 0x1(0xf6c5 bytes) 2024-09-21T14:21:52.792 INFO:teuthology.orchestra.run.smithi073.stdout:2537 write 0x3e0fee8 thru 0x3e1700e (0x7127 bytes) 2024-09-21T14:21:52.794 INFO:teuthology.orchestra.run.smithi073.stdout:2538 read 0xb3f7341 thru 0xb402a44 (0xb704 bytes) 2024-09-21T14:21:52.794 INFO:teuthology.orchestra.run.smithi073.stdout:2540 write 0xb96bc74 thru 0xb96ef61 (0x32ee bytes) 2024-09-21T14:21:52.795 INFO:teuthology.orchestra.run.smithi073.stdout:2541 punch from 0xb06dc7d to 0xb079bf7, (0xbf7a bytes) 2024-09-21T14:21:52.799 INFO:teuthology.orchestra.run.smithi073.stdout:2543 trunc from 0xc38b190 to 0x8fb8e90 2024-09-21T14:21:52.801 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:52.801+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:52.814 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:52.814 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:52.814 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:52.822 INFO:teuthology.orchestra.run.smithi073.stdout:2544 write 0xd556786 thru 0xd56026d (0x9ae8 bytes) 2024-09-21T14:21:52.823 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:52.824 INFO:teuthology.orchestra.run.smithi073.stdout:2547 writesame 0xcafb63a thru 0xcb01760 data_size 0xb(0x6127 bytes) 2024-09-21T14:21:52.828 INFO:teuthology.orchestra.run.smithi073.stdout:2549 punch from 0xb17a7ff to 0xb18a246, (0xfa47 bytes) 2024-09-21T14:21:52.828 INFO:teuthology.orchestra.run.smithi073.stdout:2550 write 0x2121df7 thru 0x2131022 (0xf22c bytes) 2024-09-21T14:21:52.830 INFO:teuthology.orchestra.run.smithi073.stdout:2551 read 0x39c82a1 thru 0x39d4ef2 (0xcc52 bytes) 2024-09-21T14:21:52.831 INFO:teuthology.orchestra.run.smithi073.stdout:2553 write 0x2aef525 thru 0x2af68df (0x73bb bytes) 2024-09-21T14:21:52.832 INFO:teuthology.orchestra.run.smithi073.stdout:2554 read 0x9502a33 thru 0x950f77c (0xcd4a bytes) 2024-09-21T14:21:52.832 INFO:teuthology.orchestra.run.smithi073.stdout:2555 write 0x8b6912 thru 0x8bc850 (0x5f3f bytes) 2024-09-21T14:21:52.834 INFO:teuthology.orchestra.run.smithi073.stdout:2556 punch from 0x7693f04 to 0x76a2df0, (0xeeec bytes) 2024-09-21T14:21:52.834 INFO:teuthology.orchestra.run.smithi073.stdout:2557 write 0x4eda83b thru 0x4ee85b2 (0xdd78 bytes) 2024-09-21T14:21:52.835 INFO:teuthology.orchestra.run.smithi073.stdout:2559 writesame 0x2f2c0b3 thru 0x2f385bd data_size 0x1(0xc50b bytes) 2024-09-21T14:21:52.853 INFO:teuthology.orchestra.run.smithi073.stdout:2560 read 0xca17e38 thru 0xca24d6b (0xcf34 bytes) 2024-09-21T14:21:52.853 INFO:teuthology.orchestra.run.smithi073.stdout:2561 punch from 0xb468501 to 0xb47646b, (0xdf6a bytes) 2024-09-21T14:21:52.853 INFO:teuthology.orchestra.run.smithi073.stdout:2562 read 0x7adc55e thru 0x7add608 (0x10ab bytes) 2024-09-21T14:21:52.854 INFO:teuthology.orchestra.run.smithi073.stdout:2565 writesame 0x9b9ef93 thru 0x9ba3686 data_size 0x13(0x46f4 bytes) 2024-09-21T14:21:52.855 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:52.855 INFO:teuthology.orchestra.run.smithi073.stdout:2567 clone 21 order 21 su 65536 sc 12 2024-09-21T14:21:52.901 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:52.902+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:52.930 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498544126999999, 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:21:52.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498527485, 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.936 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:52.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:21:52.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498502434999999, 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:21:52.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498484283, 2024-09-21T14:21:52.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:52.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:21:52.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498461045999999, 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:21:52.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.955 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498439979, 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:21:52.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498423619, 2024-09-21T14:21:52.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.965 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498393339, 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.966 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498378201000001, 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:21:52.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:21:52.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498358605, 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:21:52.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.980 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498349827999998, 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:52.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:21:52.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498337466999999, 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:52.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498328050000001, 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:21:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498306858999999, 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:21:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498289751000002, 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:21:53.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498272807999999, 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:21:53.003 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:21:53.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498256429000001, 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.008 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498215069, 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.012 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:21:53.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:21:53.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.498195387999999, 2024-09-21T14:21:53.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:21:53.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:21:53.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.093306941000000004, 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:53.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:21:53.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:21:53.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:53.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.026 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:53.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:53.028 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:53.028+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:53.128 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:53.129+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:53.171 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:53.171 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:53.179 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:53.229 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:53.230+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:53.281 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:53.281 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:53.282 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:53.282 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:53.290 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:53.330 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:53.331+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:53.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:53.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:21:53.342 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:21:53.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:21:53.351 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:21:53.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:53.414 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone20 intact 2024-09-21T14:21:53.431 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:53.431+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:53.438 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:53.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 29.19920497, 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:21:53.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:21:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:53.443 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841223606, 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:21:53.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:21:53.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841206280000002, 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841188765999998, 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:53.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:53.457 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841171511999999, 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:53.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:21:53.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:21:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841143662, 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.462 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:21:53.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.466 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841122058, 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:21:53.470 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.84110506, 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.471 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:21:53.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:21:53.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:53.475 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841088039999999, 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:53.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:21:53.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:21:53.479 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841070590000001, 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:21:53.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:53.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841053452000001, 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:21:53.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.488 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841030225000001, 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:21:53.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:21:53.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.841013951000001, 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.493 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:21:53.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840989578999999, 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.497 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:53.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.501 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840967012, 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.502 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:21:53.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840950091, 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.506 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:21:53.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:53.510 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840933591999999, 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:21:53.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840912594999999, 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:53.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.840890537, 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:53.519 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:21:53.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.66681966500000001, 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:21:53.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:53.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:53.529 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:53.531 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:53.536 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:53.536 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:53.554 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:53.555 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:53.572 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:53.573+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:53.573 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:53.589 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T14:21:53.592 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:53.592 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:53.609 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:53.610 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:53.627 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:53.627 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:53.633 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:53.633+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:53.646 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:53.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:53.647 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:53.647 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 52, 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:21:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:21:53.345964+0000", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:21:48.316234+0000", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5801984, 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:21:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "jewel", 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:21:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.9900000095367432, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.9900000095367432, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.75, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.75, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:21:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "52", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 24, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 52, 2024-09-21T14:21:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.7400000095367432, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.7400000095367432, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T14:21:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 33, 2024-09-21T14:21:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:21:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:21:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:53.709 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 47, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.710 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.711 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.712 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:21:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:21:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.716 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:21:53.720 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:53.721 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:21:53.722 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:21:53.723 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:21:53.724 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:53.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:53.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.727 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.728 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:53.730 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.731 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:21:53.732 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:21:53.733 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:21:53.734 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:21:53.735 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:21:53.735 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:21:53.735 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:21:53.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:53.735 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:53.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:53.738 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:21:53.345964+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:21:48.316234+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:21:53.738 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T14:21:53.738 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:53.739 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:53.735+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:53.798 INFO:teuthology.orchestra.run.smithi073.stdout:2568 writesame 0x6dbf8e0 thru 0x6dcaabf data_size 0x10(0xb1e0 bytes) 2024-09-21T14:21:53.808 INFO:teuthology.orchestra.run.smithi073.stdout:2569 write 0x6e27379 thru 0x6e369e5 (0xf66d bytes) 2024-09-21T14:21:53.813 INFO:teuthology.orchestra.run.smithi073.stdout:2571 writesame 0x59534c5 thru 0x5958ed5 data_size 0x1(0x5a11 bytes) 2024-09-21T14:21:53.828 INFO:teuthology.orchestra.run.smithi073.stdout:2572 write 0x7a70a87 thru 0x7a71dbb (0x1335 bytes) 2024-09-21T14:21:53.833 INFO:teuthology.orchestra.run.smithi073.stdout:2573 write 0x9433294 thru 0x943f25a (0xbfc7 bytes) 2024-09-21T14:21:53.836 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:53.835+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:53.838 INFO:teuthology.orchestra.run.smithi073.stdout:2574 read 0xd17f85 thru 0xd2366e (0xb6ea bytes) 2024-09-21T14:21:53.839 INFO:teuthology.orchestra.run.smithi073.stdout:2575 read 0xa416b25 thru 0xa41cfb6 (0x6492 bytes) 2024-09-21T14:21:53.839 INFO:teuthology.orchestra.run.smithi073.stdout:2576 read 0x21269f7 thru 0x2135f18 (0xf522 bytes) 2024-09-21T14:21:53.839 INFO:teuthology.orchestra.run.smithi073.stdout:2577 write 0xd2f22ea thru 0xd2fc333 (0xa04a bytes) 2024-09-21T14:21:53.844 INFO:teuthology.orchestra.run.smithi073.stdout:2579 writesame 0x1082cc1 thru 0x1087f41 data_size 0x1(0x5281 bytes) 2024-09-21T14:21:53.864 INFO:teuthology.orchestra.run.smithi073.stdout:2580 write 0xe0f1558 thru 0xe0f2f9a (0x1a43 bytes) 2024-09-21T14:21:53.867 INFO:teuthology.orchestra.run.smithi073.stdout:2581 read 0x80eb14b thru 0x80f354a (0x8400 bytes) 2024-09-21T14:21:53.868 INFO:teuthology.orchestra.run.smithi073.stdout:2582 read 0x595a08e thru 0x5962077 (0x7fea bytes) 2024-09-21T14:21:53.868 INFO:teuthology.orchestra.run.smithi073.stdout:2583 write 0xb04e4e7 thru 0xb04ecde (0x7f8 bytes) 2024-09-21T14:21:53.872 INFO:teuthology.orchestra.run.smithi073.stdout:2584 write 0xb8831fb thru 0xb886b87 (0x398d bytes) 2024-09-21T14:21:53.874 INFO:teuthology.orchestra.run.smithi073.stdout:2585 write 0x7cdfb1e thru 0x7ceef51 (0xf434 bytes) 2024-09-21T14:21:53.881 INFO:teuthology.orchestra.run.smithi073.stdout:2586 writesame 0x26459af thru 0x26542f5 data_size 0xb(0xe947 bytes) 2024-09-21T14:21:53.889 INFO:teuthology.orchestra.run.smithi073.stdout:2587 punch from 0x8182b78 to 0x8188a64, (0x5eec bytes) 2024-09-21T14:21:53.889 INFO:teuthology.orchestra.run.smithi073.stdout:2588 read 0x1daf3d0 thru 0x1db21e3 (0x2e14 bytes) 2024-09-21T14:21:53.890 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:53.890 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:53.890 INFO:teuthology.orchestra.run.smithi073.stdout:2589 write 0xde2fcfe thru 0xde37453 (0x7756 bytes) 2024-09-21T14:21:53.893 INFO:teuthology.orchestra.run.smithi073.stdout:2590 writesame 0x8836959 thru 0x883f162 data_size 0xb(0x880a bytes) 2024-09-21T14:21:53.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:53.900 INFO:teuthology.orchestra.run.smithi073.stdout:2592 writesame 0x3141a5d thru 0x314cd24 data_size 0xc(0xb2c8 bytes) 2024-09-21T14:21:53.905 INFO:teuthology.orchestra.run.smithi073.stdout:2593 read 0x3537d4 thru 0x35456a (0xd97 bytes) 2024-09-21T14:21:53.905 INFO:teuthology.orchestra.run.smithi073.stdout:2594 write 0x87b9671 thru 0x87c03a7 (0x6d37 bytes) 2024-09-21T14:21:53.912 INFO:teuthology.orchestra.run.smithi073.stdout:2595 punch from 0x2f64020 to 0x2f6df57, (0x9f37 bytes) 2024-09-21T14:21:53.926 INFO:teuthology.orchestra.run.smithi073.stdout:2596 trunc from 0xe0f2f9b to 0x243d643 2024-09-21T14:21:53.937 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:53.936+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:53.982 INFO:teuthology.orchestra.run.smithi073.stdout:2597 writesame 0x875cd3a thru 0x8761f7e data_size 0x1(0x5245 bytes) 2024-09-21T14:21:53.994 INFO:teuthology.orchestra.run.smithi073.stdout:2598 write 0xe737c2d thru 0xe7451e9 (0xd5bd bytes) 2024-09-21T14:21:53.998 INFO:teuthology.orchestra.run.smithi073.stdout:2599 punch from 0x9b6230a to 0x9b63162, (0xe58 bytes) 2024-09-21T14:21:53.998 INFO:teuthology.orchestra.run.smithi073.stdout:2602 punch from 0x7210cb3 to 0x721f5be, (0xe90b bytes) 2024-09-21T14:21:53.998 INFO:teuthology.orchestra.run.smithi073.stdout:2603 writesame 0x4f4451e thru 0x4f48e77 data_size 0x29(0x495a bytes) 2024-09-21T14:21:54.002 INFO:teuthology.orchestra.run.smithi073.stdout:2604 trunc from 0xe7451ea to 0xd97da8a 2024-09-21T14:21:54.009 INFO:teuthology.orchestra.run.smithi073.stdout:2606 write 0x24a93df thru 0x24b10a0 (0x7cc2 bytes) 2024-09-21T14:21:54.011 INFO:teuthology.orchestra.run.smithi073.stdout:2607 read 0x90c1159 thru 0x90c6031 (0x4ed9 bytes) 2024-09-21T14:21:54.011 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.011 INFO:teuthology.orchestra.run.smithi073.stdout:2611 punch from 0x86d26f1 to 0x86d8219, (0x5b28 bytes) 2024-09-21T14:21:54.011 INFO:teuthology.orchestra.run.smithi073.stdout:2612 writesame 0x7446693 thru 0x74469a0 data_size 0x2(0x30e bytes) 2024-09-21T14:21:54.013 INFO:teuthology.orchestra.run.smithi073.stdout:2613 read 0x89f43a7 thru 0x89f4d18 (0x972 bytes) 2024-09-21T14:21:54.013 INFO:teuthology.orchestra.run.smithi073.stdout:2614 punch from 0xc423a36 to 0xc426750, (0x2d1a bytes) 2024-09-21T14:21:54.013 INFO:teuthology.orchestra.run.smithi073.stdout:2615 write 0x1fb5dde thru 0x1fbb3f4 (0x5617 bytes) 2024-09-21T14:21:54.016 INFO:teuthology.orchestra.run.smithi073.stdout:2616 trunc from 0xd97da8a to 0xcd4250e 2024-09-21T14:21:54.021 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.021 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.024 INFO:teuthology.orchestra.run.smithi073.stdout:2617 punch from 0x30c521 to 0x30cdd3, (0x8b2 bytes) 2024-09-21T14:21:54.038 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:54.038+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:54.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:54.041 INFO:teuthology.orchestra.run.smithi073.stdout:2619 trunc from 0xcd4250e to 0x506b17 2024-09-21T14:21:54.075 INFO:teuthology.orchestra.run.smithi073.stdout:2620 read 0x3694dc thru 0x369f09 (0xa2e bytes) 2024-09-21T14:21:54.075 INFO:teuthology.orchestra.run.smithi073.stdout:2622 read 0xfba58 thru 0xfc699 (0xc42 bytes) 2024-09-21T14:21:54.093 INFO:teuthology.orchestra.run.smithi073.stdout:2623 write 0xaca191d thru 0xaca2a89 (0x116d bytes) 2024-09-21T14:21:54.101 INFO:teuthology.orchestra.run.smithi073.stdout:2626 trunc from 0xaca2a8a to 0xec9ebf8 2024-09-21T14:21:54.111 INFO:teuthology.orchestra.run.smithi073.stdout:2627 trunc from 0xec9ebf8 to 0xaad8a00 2024-09-21T14:21:54.117 INFO:teuthology.orchestra.run.smithi073.stdout:2628 read 0x745df74 thru 0x7466471 (0x84fe bytes) 2024-09-21T14:21:54.126 INFO:teuthology.orchestra.run.smithi073.stdout:2629 write 0xdffba95 thru 0xdffcdd8 (0x1344 bytes) 2024-09-21T14:21:54.134 INFO:teuthology.orchestra.run.smithi073.stdout:2630 write 0xe88824b thru 0xe88bda9 (0x3b5f bytes) 2024-09-21T14:21:54.139 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:54.139+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:54.152 INFO:teuthology.orchestra.run.smithi073.stdout:2632 trunc from 0xe88bdaa to 0x2610d62 2024-09-21T14:21:54.163 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.163 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.163 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.163 INFO:teuthology.orchestra.run.smithi073.stdout:2635 read 0x8ee2ce thru 0x8fad7b (0xcaae bytes) 2024-09-21T14:21:54.164 INFO:teuthology.orchestra.run.smithi073.stdout:2636 trunc from 0x2610d62 to 0x26d3ee9 2024-09-21T14:21:54.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:54.186 INFO:teuthology.orchestra.run.smithi073.stdout:2638 write 0xb50a890 thru 0xb514bef (0xa360 bytes) 2024-09-21T14:21:54.190 INFO:teuthology.orchestra.run.smithi073.stdout:2639 write 0x5958c7a thru 0x595e56f (0x58f6 bytes) 2024-09-21T14:21:54.193 INFO:teuthology.orchestra.run.smithi073.stdout:2641 read 0x65ba3ad thru 0x65c08cb (0x651f bytes) 2024-09-21T14:21:54.200 INFO:teuthology.orchestra.run.smithi073.stdout:2644 writesame 0xd49d4db thru 0xd4aa6a8 data_size 0x29(0xd1ce bytes) 2024-09-21T14:21:54.200 INFO:teuthology.orchestra.run.smithi073.stdout:2645 writesame 0x2f4354 thru 0x2f94f3 data_size 0x10(0x51a0 bytes) 2024-09-21T14:21:54.202 INFO:teuthology.orchestra.run.smithi073.stdout:2646 punch from 0xb82ebbd to 0xb836260, (0x76a3 bytes) 2024-09-21T14:21:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:2647 read 0x78196cc thru 0x7827513 (0xde48 bytes) 2024-09-21T14:21:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:2648 read 0xb56a893 thru 0xb571968 (0x70d6 bytes) 2024-09-21T14:21:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.203 INFO:teuthology.orchestra.run.smithi073.stdout:2650 writesame 0x23fa385 thru 0x240566f data_size 0x1(0xb2eb bytes) 2024-09-21T14:21:54.216 INFO:teuthology.orchestra.run.smithi073.stdout:2651 writesame 0x4c612c thru 0x4cc65f data_size 0x66(0x6534 bytes) 2024-09-21T14:21:54.217 INFO:teuthology.orchestra.run.smithi073.stdout:2652 read 0x5cdd638 thru 0x5ce67d8 (0x91a1 bytes) 2024-09-21T14:21:54.227 INFO:teuthology.orchestra.run.smithi073.stdout:2653 writesame 0xed57b0f thru 0xed5cc11 data_size 0x1f(0x5103 bytes) 2024-09-21T14:21:54.227 INFO:teuthology.orchestra.run.smithi073.stdout:2654 punch from 0x156cdd2 to 0x157ad8d, (0xdfbb bytes) 2024-09-21T14:21:54.237 INFO:teuthology.orchestra.run.smithi073.stdout:2655 trunc from 0xed5cc12 to 0x676adf0 2024-09-21T14:21:54.239 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:54.240+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:54.247 INFO:teuthology.orchestra.run.smithi073.stdout:2656 read 0x8b0a25 thru 0x8b3164 (0x2740 bytes) 2024-09-21T14:21:54.247 INFO:teuthology.orchestra.run.smithi073.stdout:2658 read 0x19bc811 thru 0x19bd429 (0xc19 bytes) 2024-09-21T14:21:54.250 INFO:teuthology.orchestra.run.smithi073.stdout:2660 trunc from 0x676adf0 to 0x8f78fa4 2024-09-21T14:21:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:2662 read 0x8868e8c thru 0x886ae5a (0x1fcf bytes) 2024-09-21T14:21:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:2663 read 0x7b87274 thru 0x7b91945 (0xa6d2 bytes) 2024-09-21T14:21:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:2664 read 0x26edbf9 thru 0x26fd0ff (0xf507 bytes) 2024-09-21T14:21:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:2665 read 0x6e2811c thru 0x6e29724 (0x1609 bytes) 2024-09-21T14:21:54.256 INFO:teuthology.orchestra.run.smithi073.stdout:2666 write 0x489ad8a thru 0x48a91f3 (0xe46a bytes) 2024-09-21T14:21:54.260 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.267 INFO:teuthology.orchestra.run.smithi073.stdout:2668 trunc from 0x8f78fa4 to 0xdf009b5 2024-09-21T14:21:54.272 INFO:teuthology.orchestra.run.smithi073.stdout:2669 write 0x81961c3 thru 0x81a0a57 (0xa895 bytes) 2024-09-21T14:21:54.277 INFO:teuthology.orchestra.run.smithi073.stdout:2674 write 0x9163d9a thru 0x9171d94 (0xdffb bytes) 2024-09-21T14:21:54.281 INFO:teuthology.orchestra.run.smithi073.stdout:2675 writesame 0xb32e4ff thru 0xb33df79 data_size 0x1(0xfa7b bytes) 2024-09-21T14:21:54.307 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.307 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.319 INFO:teuthology.orchestra.run.smithi073.stdout:2676 trunc from 0xdf009b5 to 0x8104e40 2024-09-21T14:21:54.321 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:54.335 INFO:teuthology.orchestra.run.smithi073.stdout:2678 read 0x48bc42d thru 0x48c6ecb (0xaa9f bytes) 2024-09-21T14:21:54.335 INFO:teuthology.orchestra.run.smithi073.stdout:2679 read 0x134ce7c thru 0x135cd74 (0xfef9 bytes) 2024-09-21T14:21:54.335 INFO:teuthology.orchestra.run.smithi073.stdout:2680 read 0x35e0983 thru 0x35eb535 (0xabb3 bytes) 2024-09-21T14:21:54.335 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.338 INFO:teuthology.orchestra.run.smithi073.stdout:2683 trunc from 0x8104e40 to 0x4fb62aa 2024-09-21T14:21:54.340 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:54.340+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:54.350 INFO:teuthology.orchestra.run.smithi073.stdout:2684 punch from 0x1cddb84 to 0x1cdeac3, (0xf3f bytes) 2024-09-21T14:21:54.350 INFO:teuthology.orchestra.run.smithi073.stdout:2685 punch from 0x5e40d4 to 0x5f0d39, (0xcc65 bytes) 2024-09-21T14:21:54.350 INFO:teuthology.orchestra.run.smithi073.stdout:2686 write 0x4c7fb97 thru 0x4c8d766 (0xdbd0 bytes) 2024-09-21T14:21:54.361 INFO:teuthology.orchestra.run.smithi073.stdout:2689 trunc from 0x4fb62aa to 0xbed741c 2024-09-21T14:21:54.366 INFO:teuthology.orchestra.run.smithi073.stdout:2692 writesame 0x7a8a3d7 thru 0x7a8e7da data_size 0x3(0x4404 bytes) 2024-09-21T14:21:54.371 INFO:teuthology.orchestra.run.smithi073.stdout:2693 write 0x491441c thru 0x4916376 (0x1f5b bytes) 2024-09-21T14:21:54.373 INFO:teuthology.orchestra.run.smithi073.stdout:2694 write 0xa24291 thru 0xa282a1 (0x4011 bytes) 2024-09-21T14:21:54.375 INFO:teuthology.orchestra.run.smithi073.stdout:2695 writesame 0xa73e421 thru 0xa74c9c8 data_size 0x4(0xe5a8 bytes) 2024-09-21T14:21:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.388 INFO:teuthology.orchestra.run.smithi073.stdout:2699 write 0xc698e64 thru 0xc6a5b75 (0xcd12 bytes) 2024-09-21T14:21:54.392 INFO:teuthology.orchestra.run.smithi073.stdout:2700 read 0xbcf904c thru 0xbd05373 (0xc328 bytes) 2024-09-21T14:21:54.392 INFO:teuthology.orchestra.run.smithi073.stdout:2703 write 0xaafaafa thru 0xab09972 (0xee79 bytes) 2024-09-21T14:21:54.397 INFO:teuthology.orchestra.run.smithi073.stdout:2704 write 0x43fce87 thru 0x4406c26 (0x9da0 bytes) 2024-09-21T14:21:54.400 INFO:teuthology.orchestra.run.smithi073.stdout:2705 writesame 0xb2dec0e thru 0xb2e3368 data_size 0x1(0x475b bytes) 2024-09-21T14:21:54.409 INFO:teuthology.orchestra.run.smithi073.stdout:2706 read 0xa57493f thru 0xa57cc28 (0x82ea bytes) 2024-09-21T14:21:54.409 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:54.409 INFO:teuthology.orchestra.run.smithi073.stdout:2708 punch from 0xa691b08 to 0xa69f49b, (0xd993 bytes) 2024-09-21T14:21:54.409 INFO:teuthology.orchestra.run.smithi073.stdout:2709 write 0x7b55606 thru 0x7b58fc5 (0x39c0 bytes) 2024-09-21T14:21:54.412 INFO:teuthology.orchestra.run.smithi073.stdout:2710 punch from 0x844cc1f to 0x844ddf0, (0x11d1 bytes) 2024-09-21T14:21:54.412 INFO:teuthology.orchestra.run.smithi073.stdout:2711 read 0x753f99d thru 0x754ec96 (0xf2fa bytes) 2024-09-21T14:21:54.412 INFO:teuthology.orchestra.run.smithi073.stdout:2712 writesame 0x75d0b65 thru 0x75d8080 data_size 0xa(0x751c bytes) 2024-09-21T14:21:54.415 INFO:teuthology.orchestra.run.smithi073.stdout:2713 clone 22 order 20 su 16384 sc 5 2024-09-21T14:21:54.441 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:54.442+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:54.445 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.445 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:54.542 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:54.543+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:54.596 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.596 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:54.644 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:54.644+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:54.745 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:54.745+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:54.747 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.747 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:54.846 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:54.846+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:54.899 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:54.899 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:54.908 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:54.949 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:54.949+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:55.047 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:55.047+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.057 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:55.149 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:55.149+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:55.199 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.199 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:55.250 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:55.250+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:55.311 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.311 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.318 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:55.363 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:55.363+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:55.366 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:21:55.426 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone21 from 0xc6a5b76 (overlap 0x506b17) to 0x202ad6 2024-09-21T14:21:55.441 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.441 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.451 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:55.451+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:55.453 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:55.526 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T14:21:55.551 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:55.552+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:55.599 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.599 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:55.652 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:55.652+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:55.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:21:55.749 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.749 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.753 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:55.754+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:55.757 INFO:teuthology.orchestra.run.smithi073.stdout:2715 punch from 0x666c29b to 0x667a329, (0xe08e bytes) 2024-09-21T14:21:55.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:55.761 INFO:teuthology.orchestra.run.smithi073.stdout:2716 read 0xbcb117c thru 0xbcbae8a (0x9d0f bytes) 2024-09-21T14:21:55.761 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:55.761 INFO:teuthology.orchestra.run.smithi073.stdout:2718 write 0x5e93b65 thru 0x5ea16d2 (0xdb6e bytes) 2024-09-21T14:21:55.776 INFO:teuthology.orchestra.run.smithi073.stdout:2719 read 0x702cf31 thru 0x7033f62 (0x7032 bytes) 2024-09-21T14:21:55.776 INFO:teuthology.orchestra.run.smithi073.stdout:2720 clone 23 order 21 su 131072 sc 3 2024-09-21T14:21:55.854 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:55.854+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:55.863 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:55.863 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:55.879 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:55.879 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:55.895 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:55.895 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:55.901 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:55.902 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:55.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:55.911 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:55.911 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:55.926 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:55.926 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:55.943 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:55.944 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:55.955 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:55.955+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:55.961 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:55.961 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:55.977 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:55.977 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:56.015 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.015 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:56.015 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:56.015 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:56.015 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.024 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:21:56.065 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:56.065+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:56.157 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:56.157+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:56.168 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.168 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:56.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458093397999999, 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:21:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:21:56.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:21:56.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458082088000001, 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.174 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:21:56.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458066712000001, 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:21:56.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:56.182 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458049184, 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:21:56.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458030232999999, 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:21:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458013853000001, 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:21:56.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:21:56.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.458004625000001, 2024-09-21T14:21:56.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:21:56.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:21:56.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457993229, 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:21:56.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457970501999998, 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:21:56.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:21:56.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457954246, 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:21:56.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:21:56.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:21:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457939491000001, 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:21:56.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:21:56.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457930437000002, 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:21:56.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:56.221 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457919393000001, 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:21:56.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457903991999999, 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:21:56.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457887293999999, 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:21:56.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.234 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457871687000001, 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:21:56.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457854565000002, 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:21:56.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:21:56.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457835954, 2024-09-21T14:21:56.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:21:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:21:56.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457816623999999, 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:21:56.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.457796986999998, 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.252 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:21:56.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:21:56.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.256 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:21:56.258 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:56.258+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:56.358 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:56.358+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:56.361 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:21:56.405 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:56.406 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:21:56.436 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone22 from 0xc6a5b76 (overlap 0xc6a5b76) to 0xb2c8583 2024-09-21T14:21:56.459 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:56.459+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:56.559 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:56.560+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:56.561 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.561 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:56.561 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:56.561 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:56.561 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:21:56.632 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T14:21:56.638 INFO:teuthology.orchestra.run.smithi073.stdout:2721 read 0x9cf962b thru 0x9cfa7a8 (0x117e bytes) 2024-09-21T14:21:56.650 INFO:teuthology.orchestra.run.smithi073.stdout:2722 trunc from 0xc6a5b76 to 0x248c7af 2024-09-21T14:21:56.662 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:56.662+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:56.668 INFO:teuthology.orchestra.run.smithi073.stdout:2725 punch from 0x1cc0173 to 0x1cc9118, (0x8fa5 bytes) 2024-09-21T14:21:56.668 INFO:teuthology.orchestra.run.smithi073.stdout:2727 read 0x104cecd thru 0x104f3e2 (0x2516 bytes) 2024-09-21T14:21:56.668 INFO:teuthology.orchestra.run.smithi073.stdout:2728 punch from 0x935ae4 to 0x93b92c, (0x5e48 bytes) 2024-09-21T14:21:56.668 INFO:teuthology.orchestra.run.smithi073.stdout:2729 punch from 0x1ee908f to 0x1eecd1f, (0x3c90 bytes) 2024-09-21T14:21:56.669 INFO:teuthology.orchestra.run.smithi073.stdout:2730 read 0x7026f1 thru 0x70db19 (0xb429 bytes) 2024-09-21T14:21:56.679 INFO:teuthology.orchestra.run.smithi073.stdout:2731 write 0x5fe479f thru 0x5fe4bd9 (0x43b bytes) 2024-09-21T14:21:56.689 INFO:teuthology.orchestra.run.smithi073.stdout:2732 writesame 0x6e98745 thru 0x6ea77a3 data_size 0x1f(0xf05f bytes) 2024-09-21T14:21:56.695 INFO:teuthology.orchestra.run.smithi073.stdout:2733 write 0x84220a4 thru 0x84291e2 (0x713f bytes) 2024-09-21T14:21:56.708 INFO:teuthology.orchestra.run.smithi073.stdout:2735 write 0xc1e04b9 thru 0xc1ebf8e (0xbad6 bytes) 2024-09-21T14:21:56.712 INFO:teuthology.orchestra.run.smithi073.stdout:2736 read 0x7834457 thru 0x7836672 (0x221c bytes) 2024-09-21T14:21:56.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.712 INFO:teuthology.orchestra.run.smithi073.stdout:2739 punch from 0x695a60d to 0x696861a, (0xe00d bytes) 2024-09-21T14:21:56.712 INFO:teuthology.orchestra.run.smithi073.stdout:2740 read 0x9ca48e2 thru 0x9cb3c2c (0xf34b bytes) 2024-09-21T14:21:56.712 INFO:teuthology.orchestra.run.smithi073.stdout:2741 writesame 0xaba8718 thru 0xabad427 data_size 0x48(0x4d10 bytes) 2024-09-21T14:21:56.716 INFO:teuthology.orchestra.run.smithi073.stdout:2743 writesame 0x89ba7a6 thru 0x89bc5b0 data_size 0x1(0x1e0b bytes) 2024-09-21T14:21:56.721 INFO:teuthology.orchestra.run.smithi073.stdout:2745 write 0x39d7333 thru 0x39e5c61 (0xe92f bytes) 2024-09-21T14:21:56.722 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290428058, 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:21:56.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:56.727 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290411416000001, 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:21:56.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290386366, 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.732 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:21:56.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:56.736 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290368214000001, 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:21:56.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.740 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290344977, 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:21:56.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:21:56.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290323910000001, 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.745 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:21:56.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290307550000001, 2024-09-21T14:21:56.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:21:56.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:21:56.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290277270000001, 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.754 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:21:56.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290262131999999, 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290242536000001, 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:21:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:21:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290233758999999, 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:21:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290221398, 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290211980999999, 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:21:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.29019079, 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:21:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290173681999999, 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290156739, 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290140359999999, 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:21:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:21:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290099000000001, 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:21:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.290079319, 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.8851908719999999, 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:21:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:21:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:56.812 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2746 read 0x1ad38cc thru 0x1adba9c (0x81d1 bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2749 trunc from 0xc1ebf8f to 0x551a7e8 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2750 write 0xce34bf5 thru 0xce3c399 (0x77a5 bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2751 trunc from 0xce3c39a to 0xc4e8954 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2753 write 0x9f88454 thru 0x9f8e95a (0x6507 bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2755 trunc from 0xc4e8954 to 0xe650a85 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2758 read 0xdf40421 thru 0xdf421ef (0x1dcf bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2759 write 0x3d1397d thru 0x3d156c0 (0x1d44 bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2760 read 0x65300bf thru 0x6533be8 (0x3b2a bytes) 2024-09-21T14:21:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:2762 read 0x961f58f thru 0x962bd62 (0xc7d4 bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2763 write 0xd754e5e thru 0xd75d504 (0x86a7 bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2766 read 0x7dd38ae thru 0x7dd9589 (0x5cdc bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2767 read 0x587867a thru 0x587ba68 (0x33ef bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2768 read 0x9deaec7 thru 0x9df05cf (0x5709 bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2769 write 0xb8d55bd thru 0xb8d8d57 (0x379b bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2770 write 0x6bb600 thru 0x6c1779 (0x617a bytes) 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:2773 read 0x5c4f16e thru 0x5c5b6dc (0xc56f bytes) 2024-09-21T14:21:56.816 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:56.813+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:56.825 INFO:teuthology.orchestra.run.smithi073.stdout:2775 trunc from 0xe650a85 to 0x353a470 2024-09-21T14:21:56.854 INFO:teuthology.orchestra.run.smithi073.stdout:2777 writesame 0xebdd57f thru 0xebe5501 data_size 0x5d(0x7f83 bytes) 2024-09-21T14:21:56.854 INFO:teuthology.orchestra.run.smithi073.stdout:2778 write 0xa58cf9c thru 0xa598675 (0xb6da bytes) 2024-09-21T14:21:56.858 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.858 INFO:teuthology.orchestra.run.smithi073.stdout:2780 write 0x8bae824 thru 0x8bbd5b2 (0xed8f bytes) 2024-09-21T14:21:56.860 INFO:teuthology.orchestra.run.smithi073.stdout:2781 writesame 0xde987bc thru 0xdea1278 data_size 0x1(0x8abd bytes) 2024-09-21T14:21:56.875 INFO:teuthology.orchestra.run.smithi073.stdout:2782 write 0x4576062 thru 0x457ee41 (0x8de0 bytes) 2024-09-21T14:21:56.878 INFO:teuthology.orchestra.run.smithi073.stdout:2783 writesame 0x5d25b2a thru 0x5d3068e data_size 0x11(0xab65 bytes) 2024-09-21T14:21:56.882 INFO:teuthology.orchestra.run.smithi073.stdout:2784 read 0x3adb751 thru 0x3ae7499 (0xbd49 bytes) 2024-09-21T14:21:56.882 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.882 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:56.882 INFO:teuthology.orchestra.run.smithi073.stdout:2787 writesame 0xb221d4c thru 0xb226c1b data_size 0x34(0x4ed0 bytes) 2024-09-21T14:21:56.885 INFO:teuthology.orchestra.run.smithi073.stdout:2788 write 0x63fd5fa thru 0x640770d (0xa114 bytes) 2024-09-21T14:21:56.890 INFO:teuthology.orchestra.run.smithi073.stdout:2790 punch from 0x5980c77 to 0x59866cb, (0x5a54 bytes) 2024-09-21T14:21:56.890 INFO:teuthology.orchestra.run.smithi073.stdout:2791 clone 24 order 20 su 32768 sc 8 2024-09-21T14:21:56.914 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:56.914+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:56.959 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:56.959 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:56.959 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:21:56.959 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:56.967 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:21:57.052 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:57.052+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:57.110 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:57.110 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:21:57.110 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:21:57.110 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:21:57.110 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:57.115 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:57.116+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:57.118 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:21:57.217 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:57.217+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:21:57.263 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 33.024097415, 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.265 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:21:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666116050999999, 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.269 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666098725000001, 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:21:57.273 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666081211000002, 2024-09-21T14:21:57.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:21:57.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.280 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666063956999999, 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.281 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:21:57.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:21:57.284 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666036107, 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.285 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:57.286 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.288 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.289 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.666014503, 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.290 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.291 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:21:57.293 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665997505, 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.294 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.295 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.296 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665980484999999, 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.299 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.300 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:21:57.301 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:21:57.302 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665963035000001, 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:57.304 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.306 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.307 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665945897, 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.308 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:21:57.310 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.311 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.66592267, 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.312 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:21:57.313 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:21:57.315 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665906396, 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.317 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.318 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.320 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665882023999998, 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:21:57.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.322 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.323 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665859457, 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:21:57.325 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.326 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:21:57.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665842536, 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:21:57.331 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.332 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665826036999999, 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:21:57.334 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.335 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.336 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665805039999999, 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:21:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.665782982, 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:21:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:21:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:21:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.4917121099999999, 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:21:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:21:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:21:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:21:57.353 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:21:57.353 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:21:57.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:21:57.355 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:57.354+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:57.455 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:21:57.455+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:21:57.462 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone23 from 0xebe5502 (overlap 0x248c7af) to 0x0 2024-09-21T14:21:57.491 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:57.491 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:57.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51002 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486207+0000", 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0032720430000000001, 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032980520000000001, 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51002 2024-09-21T14:21:57.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486207+0000", 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486207+0000", 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486207+0000", 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486209+0000", 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3609999999999999e-06 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486217+0000", 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7500000000000003e-06 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486221+0000", 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.495 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51006 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486225+0000", 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.003253756, 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033453430000000002, 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51006 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486225+0000", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486225+0000", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486225+0000", 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486227+0000", 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2500000000000001e-06 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486234+0000", 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8090000000000004e-06 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486238+0000", 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3839999999999999e-06 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51018 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486382+0000", 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0030968570000000002, 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032457620000000001, 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51018 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486382+0000", 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486382+0000", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486382+0000", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486385+0000", 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1370000000000002e-06 2024-09-21T14:21:57.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486393+0000", 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486397+0000", 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7389999999999999e-06 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.502 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51020 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486403+0000", 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0030761400000000002, 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032810069999999998, 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51020 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486403+0000", 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486403+0000", 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486403+0000", 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486405+0000", 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6929999999999999e-06 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486411+0000", 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.117e-06 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486416+0000", 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6e-06 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51027 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486437+0000", 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0030421810000000001, 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033026040000000002, 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51027 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486437+0000", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486437+0000", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486436+0000", 2024-09-21T14:21:57.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486439+0000", 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.396e-06 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486445+0000", 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9290000000000003e-06 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486449+0000", 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6700000000000002e-06 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51032 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486470+0000", 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0030088720000000001, 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033265859999999999, 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.510 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51032 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486470+0000", 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486470+0000", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486470+0000", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486472+0000", 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0669999999999999e-06 2024-09-21T14:21:57.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486478+0000", 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.049e-06 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486482+0000", 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2189999999999998e-06 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51034 3.f 3:f0c39040:::rbd_data.1121c4f3f6c2.000000000000000a:19 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486487+0000", 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.002991723, 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003370137, 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51034 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486487+0000", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486487+0000", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486487+0000", 2024-09-21T14:21:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486504+0000", 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7520000000000002e-05 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486512+0000", 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1389999999999998e-06 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486520+0000", 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7760000000000003e-06 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489782+0000", 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032613799999999999 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489853+0000", 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0896999999999996e-05 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51035 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486525+0000", 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0029538490000000001, 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003400831, 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51035 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486525+0000", 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486525+0000", 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486525+0000", 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486527+0000", 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8090000000000001e-06 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486533+0000", 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.772e-06 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486538+0000", 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3420000000000001e-06 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.521 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51036 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486542+0000", 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0029372629999999999, 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0034473239999999999, 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51036 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486542+0000", 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486542+0000", 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486541+0000", 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486544+0000", 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2910000000000002e-06 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486550+0000", 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.896e-06 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486562+0000", 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2928999999999999e-05 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51037 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486567+0000", 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0029121189999999999, 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0034799739999999998, 2024-09-21T14:21:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51037 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486567+0000", 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486567+0000", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486566+0000", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486569+0000", 2024-09-21T14:21:57.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7300000000000001e-06 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486575+0000", 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486580+0000", 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0019999999999997e-06 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:50656 3.15 3:a9da3136:::rbd_object_map.112129d42ca1.000000000000001b:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.484543+0000", 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0049358500000000003, 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0055708720000000002, 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 50656 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484543+0000", 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484543+0000", 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484542+0000", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484548+0000", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3820000000000003e-06 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484558+0000", 2024-09-21T14:21:57.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0329999999999999e-05 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484565+0000", 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0210000000000002e-06 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484608+0000", 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2750000000000002e-05 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.484690+0000", 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2117e-05 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6", 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.485034+0000", 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000343782 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:21:57.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.485956+0000", 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092192000000000001 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51039 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.486585+0000", 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0028944090000000001, 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036069090000000002, 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.535 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51039 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486585+0000", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486585+0000", 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486584+0000", 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486587+0000", 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3439999999999999e-06 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486593+0000", 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0009999999999998e-06 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.486597+0000", 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51173 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489352+0000", 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.00012726099999999999, 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000898893, 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51173 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.539 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489352+0000", 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489352+0000", 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489350+0000", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489358+0000", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8220000000000007e-06 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489369+0000", 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0668e-05 2024-09-21T14:21:57.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489377+0000", 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1089999999999998e-06 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51174 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489384+0000", 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4649999999999997e-05, 2024-09-21T14:21:57.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092076400000000002, 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51174 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489384+0000", 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489384+0000", 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489384+0000", 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489387+0000", 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.728e-06 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489393+0000", 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1090000000000003e-06 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489397+0000", 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7690000000000004e-06 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51175 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489402+0000", 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.7520000000000003e-05, 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00095691199999999998, 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51175 2024-09-21T14:21:57.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489402+0000", 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489402+0000", 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489401+0000", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489403+0000", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489409+0000", 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7509999999999997e-06 2024-09-21T14:21:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489413+0000", 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2189999999999998e-06 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51181 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489418+0000", 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.0732000000000001e-05, 2024-09-21T14:21:57.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099835399999999991, 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51181 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489418+0000", 2024-09-21T14:21:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489418+0000", 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489418+0000", 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489420+0000", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.187e-06 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489426+0000", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5500000000000002e-06 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489430+0000", 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3059999999999999e-06 2024-09-21T14:21:57.552 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51182 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489434+0000", 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.5061000000000001e-05, 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010356950000000001, 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.553 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51182 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489434+0000", 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489434+0000", 2024-09-21T14:21:57.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489434+0000", 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489436+0000", 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489441+0000", 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5489999999999997e-06 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489445+0000", 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9879999999999998e-06 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51183 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489449+0000", 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 2.9680999999999999e-05, 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010813540000000001, 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51183 2024-09-21T14:21:57.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489449+0000", 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489449+0000", 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489449+0000", 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489451+0000", 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.902e-06 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489456+0000", 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.519e-06 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489461+0000", 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5310000000000003e-06 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.560 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51187 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489465+0000", 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.4022000000000001e-05, 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001120366, 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51187 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489465+0000", 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489465+0000", 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489465+0000", 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489467+0000", 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9740000000000001e-06 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489473+0000", 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0430000000000004e-06 2024-09-21T14:21:57.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489477+0000", 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5349999999999998e-06 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51189 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489482+0000", 2024-09-21T14:21:57.564 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999967, 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011556839999999999, 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51189 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489482+0000", 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489482+0000", 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489482+0000", 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489484+0000", 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2850000000000001e-06 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489490+0000", 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:21:57.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489494+0000", 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6959999999999996e-06 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51191 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489498+0000", 2024-09-21T14:21:57.568 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999809, 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011970699999999999, 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51191 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489498+0000", 2024-09-21T14:21:57.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489498+0000", 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489498+0000", 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489500+0000", 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0820000000000001e-06 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489506+0000", 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4840000000000003e-06 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489510+0000", 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1509999999999997e-06 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51193 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489513+0000", 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999657, 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001233065, 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51193 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489513+0000", 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489513+0000", 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489513+0000", 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489515+0000", 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0310000000000001e-06 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489522+0000", 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7360000000000004e-06 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489526+0000", 2024-09-21T14:21:57.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5120000000000002e-06 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51194 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489530+0000", 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999485, 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012693520000000001, 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51194 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489530+0000", 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489530+0000", 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489530+0000", 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489532+0000", 2024-09-21T14:21:57.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489538+0000", 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489542+0000", 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2400000000000001e-06 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.579 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51195 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489546+0000", 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999332, 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013070110000000001, 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51195 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489546+0000", 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489546+0000", 2024-09-21T14:21:57.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489545+0000", 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489551+0000", 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489557+0000", 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6269999999999997e-06 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489561+0000", 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4179999999999999e-06 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51198 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489565+0000", 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9999142, 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001344523, 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51198 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489565+0000", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489565+0000", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489565+0000", 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489572+0000", 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4919999999999996e-06 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489578+0000", 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3159999999999996e-06 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489582+0000", 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51202 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489586+0000", 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9998932, 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001380046, 2024-09-21T14:21:57.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51202 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489586+0000", 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489586+0000", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489586+0000", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489588+0000", 2024-09-21T14:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1880000000000001e-06 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489593+0000", 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489598+0000", 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:21:57.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51203 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489602+0000", 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.999877, 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00143209, 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51203 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489602+0000", 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489602+0000", 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489602+0000", 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489604+0000", 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489609+0000", 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.395e-06 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489613+0000", 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4689999999999999e-06 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51206 3.e 3.4a0ccf8e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489617+0000", 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9998617, 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001473221, 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51206 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489617+0000", 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489617+0000", 2024-09-21T14:21:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489617+0000", 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489619+0000", 2024-09-21T14:21:57.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.272e-06 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489625+0000", 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6319999999999996e-06 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489629+0000", 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.138e-06 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.598 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:51209 3.f 3.209c30f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.489632+0000", 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4294967295.9998465, 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001513442, 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.599 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 51209 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489632+0000", 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489632+0000", 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489632+0000", 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489634+0000", 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489640+0000", 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1480000000000003e-06 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.489645+0000", 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 29 2024-09-21T14:21:57.603 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:57.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:21:57.605 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:57.604+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:57.624 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T14:21:57.709 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:57.705+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:57.769 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:57.769 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:21:57.769 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:21:57.769 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:21:57.769 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:57.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:21:57.806 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:57.807+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:57.891 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43037 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508454219000001, 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044045162999999998, 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43037 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382704+0000", 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1880000000000001e-06 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382710+0000", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.507e-06 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382714+0000", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2030000000000002e-06 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426582+0000", 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043868468000000001 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426657+0000", 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4577000000000001e-05 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426748+0000", 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.056e-05 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.896 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43038 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 319488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508438335999999, 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044219614999999997, 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43038 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382724+0000", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5659999999999997e-06 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382729+0000", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382733+0000", 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:21:57.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426770+0000", 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044036868 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426847+0000", 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7068000000000001e-05 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426938+0000", 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0708e-05 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43039 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508392221999999, 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044367363999999999, 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.901 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43039 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:21:57.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382767+0000", 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5959999999999998e-06 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382773+0000", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6710000000000002e-06 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382779+0000", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426961+0000", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044182402000000003 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427041+0000", 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9810999999999999e-05 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427132+0000", 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1025000000000003e-05 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43040 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.905 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508373975, 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044540541000000003, 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43040 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:21:57.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382782+0000", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382784+0000", 2024-09-21T14:21:57.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0209999999999998e-06 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382790+0000", 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5130000000000004e-06 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382794+0000", 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2200000000000003e-06 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427155+0000", 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044360978000000002 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427234+0000", 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8580999999999999e-05 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427323+0000", 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9584000000000002e-05 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43041 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 354304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508358712, 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044711682000000003, 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43041 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382799+0000", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382805+0000", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7259999999999999e-06 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382810+0000", 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2910000000000001e-06 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427346+0000", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044536282000000003 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4651000000000001e-05 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427510+0000", 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9147999999999994e-05 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.913 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43042 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 364544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508341785000001, 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044890194000000001, 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43042 2024-09-21T14:21:57.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382814+0000", 2024-09-21T14:21:57.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382816+0000", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9259999999999999e-06 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382822+0000", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.733e-06 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382826+0000", 2024-09-21T14:21:57.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.929e-06 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427536+0000", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044709567999999998 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427613+0000", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7390000000000003e-05 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427705+0000", 2024-09-21T14:21:57.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1984000000000006e-05 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43043 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508326746, 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045064142000000001, 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43043 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382831+0000", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.874e-06 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382837+0000", 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8320000000000002e-06 2024-09-21T14:21:57.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382841+0000", 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1500000000000001e-06 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427728+0000", 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044886479 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427805+0000", 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.661e-05 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427894+0000", 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9488999999999996e-05 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43044 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:21:57.922 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508311393, 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045458112000000002, 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43044 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382847+0000", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9810000000000002e-06 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382853+0000", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7880000000000003e-06 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382857+0000", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9790000000000004e-06 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427917+0000", 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045060351999999998 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428174+0000", 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025663200000000002 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428303+0000", 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129701 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43045 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508296043, 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045669176999999998, 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43045 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.927 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382860+0000", 2024-09-21T14:21:57.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382862+0000", 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382868+0000", 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6500000000000001e-06 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382873+0000", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5570000000000003e-06 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428330+0000", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045456929 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428406+0000", 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5184000000000004e-05 2024-09-21T14:21:57.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428530+0000", 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124213 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43046 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508279406, 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873328999999997, 2024-09-21T14:21:57.931 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43046 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382879+0000", 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9800000000000001e-06 2024-09-21T14:21:57.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382884+0000", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5929999999999996e-06 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382888+0000", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9249999999999997e-06 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428559+0000", 2024-09-21T14:21:57.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045670193999999997 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428632+0000", 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2978999999999994e-05 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428751+0000", 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119015 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43047 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508264229, 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046080838999999998, 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43047 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.936 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:21:57.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382894+0000", 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.956e-06 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382900+0000", 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.541e-06 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382904+0000", 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0230000000000003e-06 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428777+0000", 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873239000000003 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428850+0000", 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2762000000000006e-05 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428973+0000", 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012365400000000001 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43048 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.50824929, 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046302055000000002, 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.940 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43048 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382909+0000", 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.835e-06 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382927+0000", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8580000000000002e-05 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382932+0000", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5419999999999999e-06 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429004+0000", 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046071809999999998 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429077+0000", 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3615999999999994e-05 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429209+0000", 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199699999999999 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43049 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508220918999999, 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492449999999998, 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43049 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.945 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382935+0000", 2024-09-21T14:21:57.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382937+0000", 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382943+0000", 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382947+0000", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0670000000000002e-06 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429235+0000", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046287933000000003 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429308+0000", 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2879000000000005e-05 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429428+0000", 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120269 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43050 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508205649000001, 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046740351999999999, 2024-09-21T14:21:57.949 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43050 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382968+0000", 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7289e-05 2024-09-21T14:21:57.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383094+0000", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012638999999999999 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383114+0000", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0081999999999998e-05 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429454+0000", 2024-09-21T14:21:57.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046339578999999999 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429527+0000", 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3450999999999999e-05 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429691+0000", 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016386100000000001 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43051 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507975829999999, 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046543382000000001, 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43051 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.954 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383180+0000", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383184+0000", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7510000000000002e-06 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383207+0000", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2908000000000002e-05 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383231+0000", 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3981999999999998e-05 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429717+0000", 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046485897999999998 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429724+0000", 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5039999999999996e-06 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43052 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507891732000001, 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046500398999999998, 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43052 2024-09-21T14:21:57.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383264+0000", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383281+0000", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6515e-05 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383287+0000", 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4439999999999996e-06 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383294+0000", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1680000000000001e-06 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429747+0000", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046453853000000003 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429765+0000", 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7918999999999999e-05 2024-09-21T14:21:57.961 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43053 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507857442000001, 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492406, 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.962 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43053 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383301+0000", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3580000000000001e-06 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383307+0000", 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9619999999999998e-06 2024-09-21T14:21:57.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383311+0000", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429785+0000", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046474161 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429792+0000", 2024-09-21T14:21:57.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.139e-06 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43054 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507841665000001, 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046502169000000003, 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.966 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43054 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:21:57.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383317+0000", 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9709999999999998e-06 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383322+0000", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5589999999999996e-06 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383328+0000", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046481947000000003 2024-09-21T14:21:57.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429817+0000", 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43055 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507824569, 2024-09-21T14:21:57.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046513090999999999, 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43055 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383337+0000", 2024-09-21T14:21:57.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383342+0000", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4559999999999999e-06 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383347+0000", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3830000000000002e-06 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429839+0000", 2024-09-21T14:21:57.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492377000000001 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429845+0000", 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1689999999999997e-06 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43056 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:21:57.974 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.507806217000001, 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046522758999999997, 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43056 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383352+0000", 2024-09-21T14:21:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383357+0000", 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5380000000000002e-06 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383362+0000", 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2629999999999997e-06 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429865+0000", 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046503567000000003 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429873+0000", 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.858e-06 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:57.978 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:57.980 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2793 read 0xd408d87 thru 0xd416c0c (0xde86 bytes) 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2794 write 0xba50a7a thru 0xba54a5a (0x3fe1 bytes) 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2796 write 0xa193b78 thru 0xa1a374f (0xfbd8 bytes) 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2799 read 0x9d772fa thru 0x9d83af5 (0xc7fc bytes) 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2800 write 0xd48b21e thru 0xd48ddca (0x2bad bytes) 2024-09-21T14:21:57.981 INFO:teuthology.orchestra.run.smithi073.stdout:2801 punch from 0x12e4987 to 0x12eb140, (0x67b9 bytes) 2024-09-21T14:21:57.982 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:57.980+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:57.983 INFO:teuthology.orchestra.run.smithi073.stdout:2802 trunc from 0xebe5502 to 0x8389055 2024-09-21T14:21:57.988 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:21:58.010 INFO:teuthology.orchestra.run.smithi073.stdout:2803 trunc from 0x8389055 to 0x183ad65 2024-09-21T14:21:58.034 INFO:teuthology.orchestra.run.smithi073.stdout:2804 read 0xddc1a4 thru 0xdeba04 (0xf861 bytes) 2024-09-21T14:21:58.035 INFO:teuthology.orchestra.run.smithi073.stdout:2805 trunc from 0x183ad65 to 0x25835e4 2024-09-21T14:21:58.041 INFO:teuthology.orchestra.run.smithi073.stdout:2807 punch from 0x1da9919 to 0x1dad2d5, (0x39bc bytes) 2024-09-21T14:21:58.057 INFO:teuthology.orchestra.run.smithi073.stdout:2809 write 0xb8df3ed thru 0xb8ed014 (0xdc28 bytes) 2024-09-21T14:21:58.065 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.065 INFO:teuthology.orchestra.run.smithi073.stdout:2812 read 0x13cdcb8 thru 0x13d3003 (0x534c bytes) 2024-09-21T14:21:58.072 INFO:teuthology.orchestra.run.smithi073.stdout:2813 write 0xcd9d528 thru 0xcda13b7 (0x3e90 bytes) 2024-09-21T14:21:58.077 INFO:teuthology.orchestra.run.smithi073.stdout:2814 write 0x85db7bb thru 0x85e2b4a (0x7390 bytes) 2024-09-21T14:21:58.080 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:58.080+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:58.081 INFO:teuthology.orchestra.run.smithi073.stdout:2816 write 0x158e920 thru 0x1595816 (0x6ef7 bytes) 2024-09-21T14:21:58.091 INFO:teuthology.orchestra.run.smithi073.stdout:2819 write 0x4583873 thru 0x45892f7 (0x5a85 bytes) 2024-09-21T14:21:58.108 INFO:teuthology.orchestra.run.smithi073.stdout:2820 trunc from 0xcda13b8 to 0x209504a 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:55075 3.1f e55/19 3:f941faa4:::rbd_data.1121803c9272.0000000000000040:head v 55'196, mlcod=55'194)", 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:58.112272+0000", 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.00069443599999999999, 2024-09-21T14:21:58.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072862399999999998, 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112272+0000", 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112272+0000", 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112271+0000", 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112277+0000", 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8270000000000003e-06 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112289+0000", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2123000000000001e-05 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112295+0000", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.767e-06 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112335+0000", 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0364000000000002e-05 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:58.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:58.112354+0000", 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8660000000000001e-05 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:21:58.116 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:58.119 INFO:teuthology.orchestra.run.smithi073.stdout:2821 punch from 0x113a5f7 to 0x1144b3d, (0xa546 bytes) 2024-09-21T14:21:58.119 INFO:teuthology.orchestra.run.smithi073.stdout:2822 read 0x10e2979 thru 0x10e743d (0x4ac5 bytes) 2024-09-21T14:21:58.120 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:21:58.123 INFO:teuthology.orchestra.run.smithi073.stdout:2824 trunc from 0x209504a to 0x47d12a0 2024-09-21T14:21:58.143 INFO:teuthology.orchestra.run.smithi073.stdout:2825 write 0xd49310d thru 0xd49d142 (0xa036 bytes) 2024-09-21T14:21:58.147 INFO:teuthology.orchestra.run.smithi073.stdout:2826 punch from 0x2e0bfe2 to 0x2e0ead2, (0x2af0 bytes) 2024-09-21T14:21:58.147 INFO:teuthology.orchestra.run.smithi073.stdout:2828 write 0x782c754 thru 0x78336cd (0x6f7a bytes) 2024-09-21T14:21:58.163 INFO:teuthology.orchestra.run.smithi073.stdout:2829 trunc from 0xd49d143 to 0x44a4044 2024-09-21T14:21:58.168 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:21:58.169 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:21:58.173 INFO:teuthology.orchestra.run.smithi073.stdout:2830 write 0x27838be thru 0x27847c3 (0xf06 bytes) 2024-09-21T14:21:58.182 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:58.182+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:58.187 INFO:teuthology.orchestra.run.smithi073.stdout:2831 trunc from 0x44a4044 to 0xd5e8f1b 2024-09-21T14:21:58.187 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:21:58.188 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:21:58.202 INFO:teuthology.orchestra.run.smithi073.stdout:2832 trunc from 0xd5e8f1b to 0x486f6a1 2024-09-21T14:21:58.207 INFO:teuthology.orchestra.run.smithi073.stdout:2834 read 0x4557da9 thru 0x4560ff3 (0x924b bytes) 2024-09-21T14:21:58.218 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:21:58.219 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:21:58.223 INFO:teuthology.orchestra.run.smithi073.stdout:2835 write 0xd8a0c64 thru 0xd8a19e5 (0xd82 bytes) 2024-09-21T14:21:58.227 INFO:teuthology.orchestra.run.smithi073.stdout:2836 read 0x899b43f thru 0x899ec38 (0x37fa bytes) 2024-09-21T14:21:58.227 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.227 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.227 INFO:teuthology.orchestra.run.smithi073.stdout:2840 writesame 0xa09a9ea thru 0xa0a6311 data_size 0x9e(0xb928 bytes) 2024-09-21T14:21:58.231 INFO:teuthology.orchestra.run.smithi073.stdout:2841 punch from 0x799d3f3 to 0x79a3f4a, (0x6b57 bytes) 2024-09-21T14:21:58.239 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:21:58.239 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:21:58.240 INFO:teuthology.orchestra.run.smithi073.stdout:2842 trunc from 0xd8a19e6 to 0x55bc216 2024-09-21T14:21:58.251 INFO:teuthology.orchestra.run.smithi073.stdout:2843 read 0x139cfdf thru 0x139ea76 (0x1a98 bytes) 2024-09-21T14:21:58.251 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.251 INFO:teuthology.orchestra.run.smithi073.stdout:2846 read 0x870e6 thru 0x8de0c (0x6d27 bytes) 2024-09-21T14:21:58.258 INFO:teuthology.orchestra.run.smithi073.stdout:2847 trunc from 0x55bc216 to 0x11c5d8a 2024-09-21T14:21:58.262 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:21:58.262 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:21:58.282 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:58.282+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:58.290 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:58.290 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:58.295 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:21:58.295 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:21:58.297 INFO:teuthology.orchestra.run.smithi073.stdout:2848 clone 25 order 19 su 65536 sc 10 2024-09-21T14:21:58.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:21:58.313 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:21:58.314 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:21:58.329 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:21:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:21:58.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:21:58.382 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:58.383+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:58.424 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:58.424 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:58.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:21:58.469 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone24 from 0x11c5d8a (overlap 0x11c5d8a) to 0x554f43 2024-09-21T14:21:58.513 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:58.513+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:58.584 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:58.584+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:58.601 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:58.601 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:58.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:21:58.624 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T14:21:58.640 INFO:teuthology.orchestra.run.smithi073.stdout:2850 trunc from 0x11c5d8a to 0xd6916a2 2024-09-21T14:21:58.649 INFO:teuthology.orchestra.run.smithi073.stdout:2851 read 0x1f26b26 thru 0x1f2baa5 (0x4f80 bytes) 2024-09-21T14:21:58.649 INFO:teuthology.orchestra.run.smithi073.stdout:2854 write 0x1f1e335 thru 0x1f1f756 (0x1422 bytes) 2024-09-21T14:21:58.651 INFO:teuthology.orchestra.run.smithi073.stdout:2855 write 0x997effc thru 0x998bb92 (0xcb97 bytes) 2024-09-21T14:21:58.656 INFO:teuthology.orchestra.run.smithi073.stdout:2856 read 0xa65dfbe thru 0xa6641ba (0x61fd bytes) 2024-09-21T14:21:58.668 INFO:teuthology.orchestra.run.smithi073.stdout:2857 trunc from 0xd6916a2 to 0x42c4ecb 2024-09-21T14:21:58.678 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.679 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.679 INFO:teuthology.orchestra.run.smithi073.stdout:2862 read 0x3a464a6 thru 0x3a4e7ad (0x8308 bytes) 2024-09-21T14:21:58.679 INFO:teuthology.orchestra.run.smithi073.stdout:2863 read 0x12963d5 thru 0x129d015 (0x6c41 bytes) 2024-09-21T14:21:58.685 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:58.685+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:58.698 INFO:teuthology.orchestra.run.smithi073.stdout:2864 writesame 0xe079fd0 thru 0xe07a4bf data_size 0x2(0x4f0 bytes) 2024-09-21T14:21:58.698 INFO:teuthology.orchestra.run.smithi073.stdout:2865 writesame 0xa6ea852 thru 0xa6f5be1 data_size 0x88(0xb390 bytes) 2024-09-21T14:21:58.704 INFO:teuthology.orchestra.run.smithi073.stdout:2866 punch from 0x4259ce6 to 0x425eb4b, (0x4e65 bytes) 2024-09-21T14:21:58.704 INFO:teuthology.orchestra.run.smithi073.stdout:2867 punch from 0xb398f88 to 0xb39d3b5, (0x442d bytes) 2024-09-21T14:21:58.704 INFO:teuthology.orchestra.run.smithi073.stdout:2868 write 0x62c1fb8 thru 0x62ccbf6 (0xac3f bytes) 2024-09-21T14:21:58.706 INFO:teuthology.orchestra.run.smithi073.stdout:2869 read 0x74f7b6f thru 0x74fe026 (0x64b8 bytes) 2024-09-21T14:21:58.717 INFO:teuthology.orchestra.run.smithi073.stdout:2870 trunc from 0xe07a4c0 to 0x637cae9 2024-09-21T14:21:58.738 INFO:teuthology.orchestra.run.smithi073.stdout:2871 writesame 0xabfb0da thru 0xabff9d4 data_size 0x11(0x48fb bytes) 2024-09-21T14:21:58.739 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:21:58.739 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:21:58.739 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:21:58.739 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:21:58.744 INFO:teuthology.orchestra.run.smithi073.stdout:2874 writesame 0xbd57fe8 thru 0xbd5c132 data_size 0x1(0x414b bytes) 2024-09-21T14:21:58.744 INFO:teuthology.orchestra.run.smithi073.stdout:2875 write 0x4b30572 thru 0x4b38c29 (0x86b8 bytes) 2024-09-21T14:21:58.749 INFO:teuthology.orchestra.run.smithi073.stdout:2876 read 0x5cb6e73 thru 0x5cc2c4a (0xbdd8 bytes) 2024-09-21T14:21:58.749 INFO:teuthology.orchestra.run.smithi073.stdout:2878 read 0x2acfefd thru 0x2ad4adb (0x4bdf bytes) 2024-09-21T14:21:58.749 INFO:teuthology.orchestra.run.smithi073.stdout:2879 write 0x1d84f35 thru 0x1d85528 (0x5f4 bytes) 2024-09-21T14:21:58.751 INFO:teuthology.orchestra.run.smithi073.stdout:2880 write 0x7e901fe thru 0x7e9cce0 (0xcae3 bytes) 2024-09-21T14:21:58.762 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:58.762 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:58.771 INFO:teuthology.orchestra.run.smithi073.stdout:2882 trunc from 0xbd5c133 to 0x86eb5aa 2024-09-21T14:21:58.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:21:58.776 INFO:teuthology.orchestra.run.smithi073.stdout:2884 write 0x7f07644 thru 0x7f15b74 (0xe531 bytes) 2024-09-21T14:21:58.790 INFO:teuthology.orchestra.run.smithi073.stdout:2885 writesame 0xc8e8026 thru 0xc8f3a7a data_size 0x1(0xba55 bytes) 2024-09-21T14:21:58.790 INFO:teuthology.orchestra.run.smithi073.stdout:2886 writesame 0x48b819c thru 0x48c1291 data_size 0xf(0x90f6 bytes) 2024-09-21T14:21:58.790 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:58.790+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:58.797 INFO:teuthology.orchestra.run.smithi073.stdout:2889 write 0x1d8cbae thru 0x1d921f0 (0x5643 bytes) 2024-09-21T14:21:58.800 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.800 INFO:teuthology.orchestra.run.smithi073.stdout:2891 read 0x2a09edd thru 0x2a13e1f (0x9f43 bytes) 2024-09-21T14:21:58.800 INFO:teuthology.orchestra.run.smithi073.stdout:2892 write 0xe67044 thru 0xe70b72 (0x9b2f bytes) 2024-09-21T14:21:58.808 INFO:teuthology.orchestra.run.smithi073.stdout:2893 write 0x8430994 thru 0x8436849 (0x5eb6 bytes) 2024-09-21T14:21:58.821 INFO:teuthology.orchestra.run.smithi073.stdout:2896 trunc from 0xc8f3a7b to 0x459ec9a 2024-09-21T14:21:58.845 INFO:teuthology.orchestra.run.smithi073.stdout:2899 write 0xc3212f3 thru 0xc323a35 (0x2743 bytes) 2024-09-21T14:21:58.848 INFO:teuthology.orchestra.run.smithi073.stdout:2900 read 0x985d64d thru 0x986cfe4 (0xf998 bytes) 2024-09-21T14:21:58.856 INFO:teuthology.orchestra.run.smithi073.stdout:2901 trunc from 0xc323a36 to 0x5167ae7 2024-09-21T14:21:58.866 INFO:teuthology.orchestra.run.smithi073.stdout:2903 punch from 0x4e47c48 to 0x4e54e50, (0xd208 bytes) 2024-09-21T14:21:58.875 INFO:teuthology.orchestra.run.smithi073.stdout:2906 write 0x8a26a4e thru 0x8a33a96 (0xd049 bytes) 2024-09-21T14:21:58.886 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:58.886+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:58.891 INFO:teuthology.orchestra.run.smithi073.stdout:2907 write 0xec4eb9f thru 0xec4f339 (0x79b bytes) 2024-09-21T14:21:58.894 INFO:teuthology.orchestra.run.smithi073.stdout:2908 punch from 0x77da640 to 0x77da755, (0x115 bytes) 2024-09-21T14:21:58.899 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:58.899 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:58.905 INFO:teuthology.orchestra.run.smithi073.stdout:2909 trunc from 0xec4f33a to 0x798161a 2024-09-21T14:21:58.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:21:58.914 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.922 INFO:teuthology.orchestra.run.smithi073.stdout:2911 write 0x97a3038 thru 0x97a809d (0x5066 bytes) 2024-09-21T14:21:58.925 INFO:teuthology.orchestra.run.smithi073.stdout:2913 write 0x3335cf9 thru 0x3336473 (0x77b bytes) 2024-09-21T14:21:58.928 INFO:teuthology.orchestra.run.smithi073.stdout:2914 punch from 0x93e67c4 to 0x93e73cd, (0xc09 bytes) 2024-09-21T14:21:58.932 INFO:teuthology.orchestra.run.smithi073.stdout:2915 trunc from 0x97a809e to 0x6a0e2a0 2024-09-21T14:21:58.941 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.946 INFO:teuthology.orchestra.run.smithi073.stdout:2917 trunc from 0x6a0e2a0 to 0x9937a19 2024-09-21T14:21:58.951 INFO:teuthology.orchestra.run.smithi073.stdout:2918 write 0x4e9cb54 thru 0x4eaa0cd (0xd57a bytes) 2024-09-21T14:21:58.956 INFO:teuthology.orchestra.run.smithi073.stdout:2921 punch from 0x6300ad to 0x634434, (0x4387 bytes) 2024-09-21T14:21:58.956 INFO:teuthology.orchestra.run.smithi073.stdout:2923 write 0x6c535ff thru 0x6c62132 (0xeb34 bytes) 2024-09-21T14:21:58.967 INFO:teuthology.orchestra.run.smithi073.stdout:2924 write 0xb521b92 thru 0xb52937e (0x77ed bytes) 2024-09-21T14:21:58.970 INFO:teuthology.orchestra.run.smithi073.stdout:2925 write 0xd2bdb9 thru 0xd39dff (0xe047 bytes) 2024-09-21T14:21:58.972 INFO:teuthology.orchestra.run.smithi073.stdout:2926 writesame 0x7ec791c thru 0x7ecc587 data_size 0x4(0x4c6c bytes) 2024-09-21T14:21:58.977 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.977 INFO:teuthology.orchestra.run.smithi073.stdout:2928 read 0x5e40d0f thru 0x5e4a3e1 (0x96d3 bytes) 2024-09-21T14:21:58.977 INFO:teuthology.orchestra.run.smithi073.stdout:2929 read 0x2914c71 thru 0x291b49b (0x682b bytes) 2024-09-21T14:21:58.981 INFO:teuthology.orchestra.run.smithi073.stdout:2931 trunc from 0xb52937f to 0x83a2f2d 2024-09-21T14:21:58.987 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:21:58.987+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:21:58.992 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:58.992 INFO:teuthology.orchestra.run.smithi073.stdout:2933 read 0x27b1ca3 thru 0x27bd6e2 (0xba40 bytes) 2024-09-21T14:21:59.005 INFO:teuthology.orchestra.run.smithi073.stdout:2934 write 0xdb7f886 thru 0xdb84947 (0x50c2 bytes) 2024-09-21T14:21:59.011 INFO:teuthology.orchestra.run.smithi073.stdout:2938 write 0x3b986e5 thru 0x3b9c3c6 (0x3ce2 bytes) 2024-09-21T14:21:59.014 INFO:teuthology.orchestra.run.smithi073.stdout:2939 punch from 0x8713f6f to 0x8718d27, (0x4db8 bytes) 2024-09-21T14:21:59.015 INFO:teuthology.orchestra.run.smithi073.stdout:2940 write 0x2c8d887 thru 0x2c962d6 (0x8a50 bytes) 2024-09-21T14:21:59.020 INFO:teuthology.orchestra.run.smithi073.stdout:2942 write 0x2b8a3b6 thru 0x2b8fe3e (0x5a89 bytes) 2024-09-21T14:21:59.023 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.023 INFO:teuthology.orchestra.run.smithi073.stdout:2944 punch from 0x7f710ee to 0x7f7d7df, (0xc6f1 bytes) 2024-09-21T14:21:59.023 INFO:teuthology.orchestra.run.smithi073.stdout:2945 punch from 0xc747784 to 0xc755af4, (0xe370 bytes) 2024-09-21T14:21:59.023 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.023 INFO:teuthology.orchestra.run.smithi073.stdout:2947 writesame 0x9e68489 thru 0x9e7659c data_size 0x56(0xe114 bytes) 2024-09-21T14:21:59.028 INFO:teuthology.orchestra.run.smithi073.stdout:2948 writesame 0x69d7801 thru 0x69e70a4 data_size 0x2(0xf8a4 bytes) 2024-09-21T14:21:59.033 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.033 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:21:59.045 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.045 INFO:teuthology.orchestra.run.smithi073.stdout:2950 punch from 0x4fed994 to 0x4ff8917, (0xaf83 bytes) 2024-09-21T14:21:59.045 INFO:teuthology.orchestra.run.smithi073.stdout:2951 write 0xc7021e6 thru 0xc70df47 (0xbd62 bytes) 2024-09-21T14:21:59.049 INFO:teuthology.orchestra.run.smithi073.stdout:2952 punch from 0xaf4e9f9 to 0xaf56573, (0x7b7a bytes) 2024-09-21T14:21:59.049 INFO:teuthology.orchestra.run.smithi073.stdout:2953 read 0x602a9c7 thru 0x6035255 (0xa88f bytes) 2024-09-21T14:21:59.049 INFO:teuthology.orchestra.run.smithi073.stdout:2954 read 0x17b6d26 thru 0x17bc111 (0x53ec bytes) 2024-09-21T14:21:59.049 INFO:teuthology.orchestra.run.smithi073.stdout:2955 punch from 0x2ea164d to 0x2ea1bf9, (0x5ac bytes) 2024-09-21T14:21:59.057 INFO:teuthology.orchestra.run.smithi073.stdout:2957 writesame 0xe3c765a thru 0xe3d5f50 data_size 0x1(0xe8f7 bytes) 2024-09-21T14:21:59.057 INFO:teuthology.orchestra.run.smithi073.stdout:2958 write 0x2db57cb thru 0x2db63fd (0xc33 bytes) 2024-09-21T14:21:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:2959 trunc from 0xe3d5f51 to 0xcdc82b8 2024-09-21T14:21:59.071 INFO:teuthology.orchestra.run.smithi073.stdout:2960 writesame 0x15abf1 thru 0x1621fc data_size 0xa(0x760c bytes) 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 56, 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:21:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:21:58.363704+0000", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:21:48.316234+0000", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5806080, 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:21:59.084 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "jewel", 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:21:59.085 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:21:59.086 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:59.087 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:59.088 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:59.089 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.090 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.9900000095367432, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.9900000095367432, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.75, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.75, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:21:59.091 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:21:59.092 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:59.093 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:59.094 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:59.095 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:59.096 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.097 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:21:59.098 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:21:59.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "56", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 28, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 56, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:21:59.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:21:59.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:21:59.102 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.7400000095367432, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.7400000095367432, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:59.103 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.104 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.106 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 33, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:21:59.114 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.115 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.116 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:21:59.117 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:21:59.118 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.119 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:21:59.125 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.126 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.127 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.129 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.131 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 47, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.132 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.133 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:21:59.134 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:21:59.135 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.138 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.139 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.140 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.141 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:21:59.142 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:21:59.143 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:21:59.148 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.150 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.151 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.153 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:21:59.154 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:21:59.155 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:21:59.156 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:21:59.157 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:21:59.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:21:59.157 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:21:59.158 INFO:teuthology.orchestra.run.smithi073.stdout:2961 write 0xe973c57 thru 0xe9820e1 (0xe48b bytes) 2024-09-21T14:21:59.158 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.158 INFO:teuthology.orchestra.run.smithi073.stdout:2963 read 0xaa6761a thru 0xaa685b2 (0xf99 bytes) 2024-09-21T14:21:59.158 INFO:teuthology.orchestra.run.smithi073.stdout:2964 writesame 0x5010cf4 thru 0x5020010 data_size 0x11(0xf31d bytes) 2024-09-21T14:21:59.159 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:21:58.363704+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:21:48.316234+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:21:59.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:21:59.160 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.160 INFO:teuthology.orchestra.run.smithi073.stdout:2967 trunc from 0xe9820e2 to 0x6465aac 2024-09-21T14:21:59.160 INFO:teuthology.orchestra.run.smithi073.stdout:2968 punch from 0x1acc646 to 0x1adb5c3, (0xef7d bytes) 2024-09-21T14:21:59.160 INFO:teuthology.orchestra.run.smithi073.stdout:2969 punch from 0x3fa494a to 0x3faf17d, (0xa833 bytes) 2024-09-21T14:21:59.161 INFO:teuthology.orchestra.run.smithi073.stdout:2970 clone 26 order 23 su 262144 sc 15 2024-09-21T14:21:59.161 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:59.158+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:59.162 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.162 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:21:59.258 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:59.258+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:59.317 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.317 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:21:59.359 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:21:59.359+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:21:59.412 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:21:59.432 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.432 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:21:59.448 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:21:59.449 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:21:59.460 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-09-21T14:21:59.381439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":734077751,"num_objects":1050,"num_object_clones":238,"num_object_copies":2100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1050,"num_whiteouts":421,"num_read":69980,"num_read_kb":320293,"num_write":9499,"num_write_kb":129255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7539,"ondisk_log_size":7539,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":275612,"kb_used_data":25004,"kb_used_omap":10,"kb_used_meta":250485,"kb_avail":660327268,"statfs":{"total":676457349120,"available":676175122432,"internally_reserved":0,"allocated":25604096,"data_stored":156401432,"data_compressed":8517068,"data_compressed_allocated":10620928,"data_compressed_original":145916635,"omap_allocated":11150,"internal_metadata":256496754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":183400058,"num_objects":137,"num_object_clones":35,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":41,"num_read":6426,"num_read_kb":17376,"num_write":886,"num_write_kb":10627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026918"},"pg_stats":[{"pgid":"3.1f","version":"54'188","reported_seq":1953,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.418890+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:21:57.418890+0000","last_peered":"2024-09-21T14:21:57.418890+0000","last_clean":"2024-09-21T14:21:57.418890+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:21:57.418890+0000","last_undegraded":"2024-09-21T14:21:57.418890+0000","last_fullsized":"2024-09-21T14:21:57.418890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":15177272,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":2398,"num_read_kb":10045,"num_write":195,"num_write_kb":1811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"55'90","reported_seq":374,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.370210+0000","last_change":"2024-09-21T14:21:49.388656+0000","last_active":"2024-09-21T14:21:57.370210+0000","last_peered":"2024-09-21T14:21:57.370210+0000","last_clean":"2024-09-21T14:21:57.370210+0000","last_became_active":"2024-09-21T14:21:49.343683+0000","last_became_peered":"2024-09-21T14:21:49.343683+0000","last_unstale":"2024-09-21T14:21:57.370210+0000","last_undegraded":"2024-09-21T14:21:57.370210+0000","last_fullsized":"2024-09-21T14:21:57.370210+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:13.419608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":8714282,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":205,"num_read_kb":5889,"num_write":87,"num_write_kb":3205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2693260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"54'114","reported_seq":744,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.399823+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:21:57.399823+0000","last_peered":"2024-09-21T14:21:57.399823+0000","last_clean":"2024-09-21T14:21:57.399823+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:21:57.399823+0000","last_undegraded":"2024-09-21T14:21:57.399823+0000","last_fullsized":"2024-09-21T14:21:57.399823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":9718972,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":626,"num_read_kb":1277,"num_write":149,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"55'160","reported_seq":3256,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.428861+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:21:57.428861+0000","last_peered":"2024-09-21T14:21:57.428861+0000","last_clean":"2024-09-21T14:21:57.428861+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:21:57.428861+0000","last_undegraded":"2024-09-21T14:21:57.428861+0000","last_fullsized":"2024-09-21T14:21:57.428861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":34818474,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":3176,"num_read_kb":28918,"num_write":160,"num_write_kb":4067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"54'178","reported_seq":2261,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.658779+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:21:56.658779+0000","last_peered":"2024-09-21T14:21:56.658779+0000","last_clean":"2024-09-21T14:21:56.658779+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:21:56.658779+0000","last_undegraded":"2024-09-21T14:21:56.658779+0000","last_fullsized":"2024-09-21T14:21:56.658779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":25691464,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":2440,"num_read_kb":22181,"num_write":148,"num_write_kb":9559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"54'201","reported_seq":2234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.432414+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:21:57.432414+0000","last_peered":"2024-09-21T14:21:57.432414+0000","last_clean":"2024-09-21T14:21:57.432414+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:21:57.432414+0000","last_undegraded":"2024-09-21T14:21:57.432414+0000","last_fullsized":"2024-09-21T14:21:57.432414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":14336233,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":2731,"num_read_kb":4366,"num_write":231,"num_write_kb":1714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"54'154","reported_seq":3545,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.770209+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:21:56.770209+0000","last_peered":"2024-09-21T14:21:56.770209+0000","last_clean":"2024-09-21T14:21:56.770209+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:21:56.770209+0000","last_undegraded":"2024-09-21T14:21:56.770209+0000","last_fullsized":"2024-09-21T14:21:56.770209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":26186554,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3897,"num_read_kb":17677,"num_write":142,"num_write_kb":1883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":838,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.370166+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:21:57.370166+0000","last_peered":"2024-09-21T14:21:57.370166+0000","last_clean":"2024-09-21T14:21:57.370166+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:21:57.370166+0000","last_undegraded":"2024-09-21T14:21:57.370166+0000","last_fullsized":"2024-09-21T14:21:57.370166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":10754607,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":745,"num_read_kb":1998,"num_write":100,"num_write_kb":1267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"54'331","reported_seq":865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.876073+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:21:56.876073+0000","last_peered":"2024-09-21T14:21:56.876073+0000","last_clean":"2024-09-21T14:21:56.876073+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:21:56.876073+0000","last_undegraded":"2024-09-21T14:21:56.876073+0000","last_fullsized":"2024-09-21T14:21:56.876073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":11195071,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1609,"num_read_kb":5268,"num_write":479,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"54'129","reported_seq":1193,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.803129+0000","last_change":"2024-09-21T14:21:52.139246+0000","last_active":"2024-09-21T14:21:56.803129+0000","last_peered":"2024-09-21T14:21:56.803129+0000","last_clean":"2024-09-21T14:21:56.803129+0000","last_became_active":"2024-09-21T14:21:52.138595+0000","last_became_peered":"2024-09-21T14:21:52.138595+0000","last_unstale":"2024-09-21T14:21:56.803129+0000","last_undegraded":"2024-09-21T14:21:56.803129+0000","last_fullsized":"2024-09-21T14:21:56.803129+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:23.662981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":7432567,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":1259,"num_read_kb":1768,"num_write":114,"num_write_kb":718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":83845,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"55'375","reported_seq":1746,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.395022+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:21:57.395022+0000","last_peered":"2024-09-21T14:21:57.395022+0000","last_clean":"2024-09-21T14:21:57.395022+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:21:57.395022+0000","last_undegraded":"2024-09-21T14:21:57.395022+0000","last_fullsized":"2024-09-21T14:21:57.395022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":20693180,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":3136,"num_read_kb":18415,"num_write":516,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"54'225","reported_seq":1286,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.432816+0000","last_change":"2024-09-21T14:21:52.140205+0000","last_active":"2024-09-21T14:21:57.432816+0000","last_peered":"2024-09-21T14:21:57.432816+0000","last_clean":"2024-09-21T14:21:57.432816+0000","last_became_active":"2024-09-21T14:21:52.138972+0000","last_became_peered":"2024-09-21T14:21:52.138972+0000","last_unstale":"2024-09-21T14:21:57.432816+0000","last_undegraded":"2024-09-21T14:21:57.432816+0000","last_fullsized":"2024-09-21T14:21:57.432816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:41.279903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":5652262,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":2210,"num_read_kb":2587,"num_write":281,"num_write_kb":1717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1075708,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.361471+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:21:56.361471+0000","last_peered":"2024-09-21T14:21:56.361471+0000","last_clean":"2024-09-21T14:21:56.361471+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:21:56.361471+0000","last_undegraded":"2024-09-21T14:21:56.361471+0000","last_fullsized":"2024-09-21T14:21:56.361471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'187","reported_seq":1617,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.803251+0000","last_change":"2024-09-21T14:21:52.138940+0000","last_active":"2024-09-21T14:21:56.803251+0000","last_peered":"2024-09-21T14:21:56.803251+0000","last_clean":"2024-09-21T14:21:56.803251+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:21:56.803251+0000","last_undegraded":"2024-09-21T14:21:56.803251+0000","last_fullsized":"2024-09-21T14:21:56.803251+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":38840058,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":18,"num_read":1952,"num_read_kb":5776,"num_write":175,"num_write_kb":2092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.364257+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:21:56.364257+0000","last_peered":"2024-09-21T14:21:56.364257+0000","last_clean":"2024-09-21T14:21:56.364257+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:21:56.364257+0000","last_undegraded":"2024-09-21T14:21:56.364257+0000","last_fullsized":"2024-09-21T14:21:56.364257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'266","reported_seq":1728,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.740501+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:21:56.740501+0000","last_peered":"2024-09-21T14:21:56.740501+0000","last_clean":"2024-09-21T14:21:56.740501+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:21:56.740501+0000","last_undegraded":"2024-09-21T14:21:56.740501+0000","last_fullsized":"2024-09-21T14:21:56.740501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":6735975,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":2448,"num_read_kb":4823,"num_write":404,"num_write_kb":963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.387908+0000","last_change":"2024-09-21T14:21:49.338538+0000","last_active":"2024-09-21T14:21:56.387908+0000","last_peered":"2024-09-21T14:21:56.387908+0000","last_clean":"2024-09-21T14:21:56.387908+0000","last_became_active":"2024-09-21T14:21:49.338272+0000","last_became_peered":"2024-09-21T14:21:49.338272+0000","last_unstale":"2024-09-21T14:21:56.387908+0000","last_undegraded":"2024-09-21T14:21:56.387908+0000","last_fullsized":"2024-09-21T14:21:56.387908+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:55.449259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"54'121","reported_seq":1598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.858274+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:21:56.858274+0000","last_peered":"2024-09-21T14:21:56.858274+0000","last_clean":"2024-09-21T14:21:56.858274+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:21:56.858274+0000","last_undegraded":"2024-09-21T14:21:56.858274+0000","last_fullsized":"2024-09-21T14:21:56.858274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":22733738,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":1473,"num_read_kb":22671,"num_write":106,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.360761+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:21:56.360761+0000","last_peered":"2024-09-21T14:21:56.360761+0000","last_clean":"2024-09-21T14:21:56.360761+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:21:56.360761+0000","last_undegraded":"2024-09-21T14:21:56.360761+0000","last_fullsized":"2024-09-21T14:21:56.360761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"54'126","reported_seq":3587,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.801983+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:21:56.801983+0000","last_peered":"2024-09-21T14:21:56.801983+0000","last_clean":"2024-09-21T14:21:56.801983+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:21:56.801983+0000","last_undegraded":"2024-09-21T14:21:56.801983+0000","last_fullsized":"2024-09-21T14:21:56.801983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":35499757,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":3493,"num_read_kb":25647,"num_write":117,"num_write_kb":5399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"54'166","reported_seq":1604,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.411533+0000","last_change":"2024-09-21T14:21:52.139305+0000","last_active":"2024-09-21T14:21:57.411533+0000","last_peered":"2024-09-21T14:21:57.411533+0000","last_clean":"2024-09-21T14:21:57.411533+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:21:57.411533+0000","last_undegraded":"2024-09-21T14:21:57.411533+0000","last_fullsized":"2024-09-21T14:21:57.411533+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":33097966,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":2046,"num_read_kb":16038,"num_write":166,"num_write_kb":4641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.364049+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:21:56.364049+0000","last_peered":"2024-09-21T14:21:56.364049+0000","last_clean":"2024-09-21T14:21:56.364049+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:21:56.364049+0000","last_undegraded":"2024-09-21T14:21:56.364049+0000","last_fullsized":"2024-09-21T14:21:56.364049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":55,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.367502+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:21:56.367502+0000","last_peered":"2024-09-21T14:21:56.367502+0000","last_clean":"2024-09-21T14:21:56.367502+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:21:56.367502+0000","last_undegraded":"2024-09-21T14:21:56.367502+0000","last_fullsized":"2024-09-21T14:21:56.367502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"54'422","reported_seq":1230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.890558+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:21:56.890558+0000","last_peered":"2024-09-21T14:21:56.890558+0000","last_clean":"2024-09-21T14:21:56.890558+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:21:56.890558+0000","last_undegraded":"2024-09-21T14:21:56.890558+0000","last_fullsized":"2024-09-21T14:21:56.890558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":9481311,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":2376,"num_read_kb":4610,"num_write":674,"num_write_kb":2036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.366029+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:21:56.366029+0000","last_peered":"2024-09-21T14:21:56.366029+0000","last_clean":"2024-09-21T14:21:56.366029+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:21:56.366029+0000","last_undegraded":"2024-09-21T14:21:56.366029+0000","last_fullsized":"2024-09-21T14:21:56.366029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"54'341","reported_seq":786,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.829782+0000","last_change":"2024-09-21T14:21:52.139498+0000","last_active":"2024-09-21T14:21:56.829782+0000","last_peered":"2024-09-21T14:21:56.829782+0000","last_clean":"2024-09-21T14:21:56.829782+0000","last_became_active":"2024-09-21T14:21:52.138744+0000","last_became_peered":"2024-09-21T14:21:52.138744+0000","last_unstale":"2024-09-21T14:21:56.829782+0000","last_undegraded":"2024-09-21T14:21:56.829782+0000","last_fullsized":"2024-09-21T14:21:56.829782+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:16:08.802429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":16225386,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":1463,"num_read_kb":5502,"num_write":527,"num_write_kb":2792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2088730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.364260+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:21:56.364260+0000","last_peered":"2024-09-21T14:21:56.364260+0000","last_clean":"2024-09-21T14:21:56.364260+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:21:56.364260+0000","last_undegraded":"2024-09-21T14:21:56.364260+0000","last_fullsized":"2024-09-21T14:21:56.364260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"54'299","reported_seq":1378,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.830599+0000","last_change":"2024-09-21T14:21:52.139320+0000","last_active":"2024-09-21T14:21:56.830599+0000","last_peered":"2024-09-21T14:21:56.830599+0000","last_clean":"2024-09-21T14:21:56.830599+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:21:56.830599+0000","last_undegraded":"2024-09-21T14:21:56.830599+0000","last_fullsized":"2024-09-21T14:21:56.830599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":36597542,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":1821,"num_read_kb":3868,"num_write":416,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.361243+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:21:56.361243+0000","last_peered":"2024-09-21T14:21:56.361243+0000","last_clean":"2024-09-21T14:21:56.361243+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:21:56.361243+0000","last_undegraded":"2024-09-21T14:21:56.361243+0000","last_fullsized":"2024-09-21T14:21:56.361243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'401","reported_seq":2317,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.394048+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:21:57.394048+0000","last_peered":"2024-09-21T14:21:57.394048+0000","last_clean":"2024-09-21T14:21:57.394048+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:21:57.394048+0000","last_undegraded":"2024-09-21T14:21:57.394048+0000","last_fullsized":"2024-09-21T14:21:57.394048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":13406217,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":4264,"num_read_kb":9386,"num_write":495,"num_write_kb":2826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"54'300","reported_seq":2217,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.830533+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:21:56.830533+0000","last_peered":"2024-09-21T14:21:56.830533+0000","last_clean":"2024-09-21T14:21:56.830533+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:21:56.830533+0000","last_undegraded":"2024-09-21T14:21:56.830533+0000","last_fullsized":"2024-09-21T14:21:56.830533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":62402778,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":3845,"num_read_kb":13053,"num_write":295,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"54'215","reported_seq":455,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.399688+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:21:57.399688+0000","last_peered":"2024-09-21T14:21:57.399688+0000","last_clean":"2024-09-21T14:21:57.399688+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:21:57.399688+0000","last_undegraded":"2024-09-21T14:21:57.399688+0000","last_fullsized":"2024-09-21T14:21:57.399688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":10759586,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":478,"num_read_kb":1554,"num_write":221,"num_write_kb":1397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"54'197","reported_seq":1305,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.867285+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:21:56.867285+0000","last_peered":"2024-09-21T14:21:56.867285+0000","last_clean":"2024-09-21T14:21:56.867285+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:21:56.867285+0000","last_undegraded":"2024-09-21T14:21:56.867285+0000","last_fullsized":"2024-09-21T14:21:56.867285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":31793791,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":1494,"num_read_kb":2810,"num_write":340,"num_write_kb":2075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"54'249","reported_seq":633,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.375681+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:21:57.375681+0000","last_peered":"2024-09-21T14:21:57.375681+0000","last_clean":"2024-09-21T14:21:57.375681+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:21:57.375681+0000","last_undegraded":"2024-09-21T14:21:57.375681+0000","last_fullsized":"2024-09-21T14:21:57.375681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":19036784,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":1297,"num_read_kb":2502,"num_write":304,"num_write_kb":1456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"55'140","reported_seq":1487,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.440332+0000","last_change":"2024-09-21T14:21:34.229265+0000","last_active":"2024-09-21T14:21:57.440332+0000","last_peered":"2024-09-21T14:21:57.440332+0000","last_clean":"2024-09-21T14:21:57.440332+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:21:57.440332+0000","last_undegraded":"2024-09-21T14:21:57.440332+0000","last_fullsized":"2024-09-21T14:21:57.440332+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:04.756321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":18390263,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":1109,"num_read_kb":8608,"num_write":136,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"53'634","reported_seq":2131,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.806613+0000","last_change":"2024-09-21T14:21:52.141637+0000","last_active":"2024-09-21T14:21:56.806613+0000","last_peered":"2024-09-21T14:21:56.806613+0000","last_clean":"2024-09-21T14:21:56.806613+0000","last_became_active":"2024-09-21T14:21:52.140890+0000","last_became_peered":"2024-09-21T14:21:52.140890+0000","last_unstale":"2024-09-21T14:21:56.806613+0000","last_undegraded":"2024-09-21T14:21:56.806613+0000","last_fullsized":"2024-09-21T14:21:56.806613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:12.353305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":30989956,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":4262,"num_read_kb":14217,"num_write":1048,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3052424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'179","reported_seq":2391,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.806638+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:21:56.806638+0000","last_peered":"2024-09-21T14:21:56.806638+0000","last_clean":"2024-09-21T14:21:56.806638+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:21:56.806638+0000","last_undegraded":"2024-09-21T14:21:56.806638+0000","last_fullsized":"2024-09-21T14:21:56.806638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":44952917,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":2510,"num_read_kb":8868,"num_write":167,"num_write_kb":14495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'182","reported_seq":1439,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.886272+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:21:56.886272+0000","last_peered":"2024-09-21T14:21:56.886272+0000","last_clean":"2024-09-21T14:21:56.886272+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:21:56.886272+0000","last_undegraded":"2024-09-21T14:21:56.886272+0000","last_fullsized":"2024-09-21T14:21:56.886272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":28690798,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1742,"num_read_kb":8943,"num_write":195,"num_write_kb":3838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"54'191","reported_seq":1320,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T14:21:57.399608+0000","last_change":"2024-09-21T14:21:52.139083+0000","last_active":"2024-09-21T14:21:57.399608+0000","last_peered":"2024-09-21T14:21:57.399608+0000","last_clean":"2024-09-21T14:21:57.399608+0000","last_became_active":"2024-09-21T14:21:52.138188+0000","last_became_peered":"2024-09-21T14:21:52.138188+0000","last_unstale":"2024-09-21T14:21:57.399608+0000","last_undegraded":"2024-09-21T14:21:57.399608+0000","last_fullsized":"2024-09-21T14:21:57.399608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:15.976782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":41040236,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1335,"num_read_kb":12270,"num_write":241,"num_write_kb":11534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":3452172,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'151","reported_seq":1003,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.829787+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:21:56.829787+0000","last_peered":"2024-09-21T14:21:56.829787+0000","last_clean":"2024-09-21T14:21:56.829787+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:21:56.829787+0000","last_undegraded":"2024-09-21T14:21:56.829787+0000","last_fullsized":"2024-09-21T14:21:56.829787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":18284751,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1420,"num_read_kb":10201,"num_write":166,"num_write_kb":5500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"54'398","reported_seq":2741,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T14:21:56.879806+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:21:56.879806+0000","last_peered":"2024-09-21T14:21:56.879806+0000","last_clean":"2024-09-21T14:21:56.879806+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:21:56.879806+0000","last_undegraded":"2024-09-21T14:21:56.879806+0000","last_fullsized":"2024-09-21T14:21:56.879806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":24156616,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":4604,"num_read_kb":18457,"num_write":543,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":733487364,"num_objects":1046,"num_object_clones":238,"num_object_copies":2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1046,"num_whiteouts":421,"num_read":69864,"num_read_kb":320193,"num_write":9338,"num_write_kb":126837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23228416,"data_stored":169516353,"data_compressed":9345400,"data_compressed_allocated":11759616,"data_compressed_original":160353014,"omap_allocated":0,"internal_metadata":0},"log_size":7419,"ondisk_log_size":7419,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2129920,"data_stored":9447802,"data_compressed":474518,"data_compressed_allocated":622592,"data_compressed_original":8482816,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38296,"kb_used_data":3088,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333544,"statfs":{"total":96636764160,"available":96597549056,"internally_reserved":0,"allocated":3162112,"data_stored":18058515,"data_compressed":1071205,"data_compressed_allocated":1294336,"data_compressed_original":16768151,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37992,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333848,"statfs":{"total":96636764160,"available":96597860352,"internally_reserved":0,"allocated":2846720,"data_stored":16229312,"data_compressed":785294,"data_compressed_allocated":1085440,"data_compressed_original":15052434,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39628,"kb_used_data":4384,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332212,"statfs":{"total":96636764160,"available":96596185088,"internally_reserved":0,"allocated":4489216,"data_stored":29097156,"data_compressed":1616936,"data_compressed_allocated":1925120,"data_compressed_original":27270466,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39708,"kb_used_data":4492,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332132,"statfs":{"total":96636764160,"available":96596103168,"internally_reserved":0,"allocated":4599808,"data_stored":29354669,"data_compressed":1609167,"data_compressed_allocated":2129920,"data_compressed_original":27516927,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":2,"up_from":0,"seq":55834574860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38624,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333216,"statfs":{"total":96636764160,"available":96597213184,"internally_reserved":0,"allocated":3493888,"data_stored":20683824,"data_compressed":1169393,"data_compressed_allocated":1449984,"data_compressed_original":19316231,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44072,"kb_used_data":4768,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327768,"statfs":{"total":96636764160,"available":96591634432,"internally_reserved":0,"allocated":4882432,"data_stored":33530154,"data_compressed":1790555,"data_compressed_allocated":2113536,"data_compressed_original":31509610,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4259840,"data_stored":33252092,"data_compressed":1790555,"data_compressed_allocated":2113536,"data_compressed_original":31509610,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2932736,"data_stored":20501234,"data_compressed":1172182,"data_compressed_allocated":1458176,"data_compressed_original":19373575,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3915776,"data_stored":28486210,"data_compressed":1576757,"data_compressed_allocated":2084864,"data_compressed_original":26943487,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3850240,"data_stored":28812887,"data_compressed":1616936,"data_compressed_allocated":1925120,"data_compressed_original":27270466,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":15423561,"data_compressed":755081,"data_compressed_allocated":1048576,"data_compressed_original":14524050,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2543616,"data_stored":17780472,"data_compressed":1071205,"data_compressed_allocated":1294336,"data_compressed_original":16768151,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1527808,"data_stored":9185182,"data_compressed":474518,"data_compressed_allocated":622592,"data_compressed_original":8482816,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:21:59.462 INFO:tasks.thrashosds.thrasher:Setting 3.d to [6, 2] 2024-09-21T14:21:59.463 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.d', '6', '2'] 2024-09-21T14:21:59.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.d 6 2 2024-09-21T14:21:59.485 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:59.485+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:59.512 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone25 from 0x6465aac (overlap 0x11c5d8a) to 0x0 2024-09-21T14:21:59.561 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:21:59.561+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:21:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:21:59.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:21:59.662+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:21:59.713 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T14:21:59.721 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.721 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:21:59.736 INFO:teuthology.orchestra.run.smithi073.stdout:2971 write 0x7f13c5a thru 0x7f1fff4 (0xc39b bytes) 2024-09-21T14:21:59.739 INFO:teuthology.orchestra.run.smithi073.stdout:2972 write 0x6c5acab thru 0x6c65e45 (0xb19b bytes) 2024-09-21T14:21:59.748 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.759 INFO:teuthology.orchestra.run.smithi073.stdout:2976 write 0xc3a531b thru 0xc3a7ee9 (0x2bcf bytes) 2024-09-21T14:21:59.762 INFO:teuthology.orchestra.run.smithi073.stdout:2977 write 0x750d357 thru 0x7512988 (0x5632 bytes) 2024-09-21T14:21:59.771 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:21:59.771+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:21:59.771 INFO:teuthology.orchestra.run.smithi073.stdout:2978 write 0x49e2605 thru 0x49ed43b (0xae37 bytes) 2024-09-21T14:21:59.779 INFO:teuthology.orchestra.run.smithi073.stdout:2979 punch from 0x3644436 to 0x36495f3, (0x51bd bytes) 2024-09-21T14:21:59.787 INFO:teuthology.orchestra.run.smithi073.stdout:2981 write 0xe790f68 thru 0xe795863 (0x48fc bytes) 2024-09-21T14:21:59.790 INFO:teuthology.orchestra.run.smithi073.stdout:2982 writesame 0xcd0b81b thru 0xcd150a2 data_size 0xc(0x9888 bytes) 2024-09-21T14:21:59.809 INFO:teuthology.orchestra.run.smithi073.stdout:2983 trunc from 0xe795864 to 0x10f0f81 2024-09-21T14:21:59.858 INFO:teuthology.orchestra.run.smithi073.stdout:2984 write 0xe89ed7 thru 0xe98232 (0xe35c bytes) 2024-09-21T14:21:59.860 INFO:teuthology.orchestra.run.smithi073.stdout:2985 read 0x1089d7d thru 0x108edd1 (0x5055 bytes) 2024-09-21T14:21:59.860 INFO:teuthology.orchestra.run.smithi073.stdout:2987 read 0x521f9b thru 0x52ac8c (0x8cf2 bytes) 2024-09-21T14:21:59.863 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:21:59.863+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:21:59.873 INFO:teuthology.orchestra.run.smithi073.stdout:2990 trunc from 0x10f0f81 to 0xbc433ef 2024-09-21T14:21:59.878 INFO:teuthology.orchestra.run.smithi073.stdout:2992 write 0x51595be thru 0x515d32b (0x3d6e bytes) 2024-09-21T14:21:59.880 INFO:teuthology.orchestra.run.smithi073.stdout:2993 read 0x7ac624e thru 0x7ad2f88 (0xcd3b bytes) 2024-09-21T14:21:59.880 INFO:teuthology.orchestra.run.smithi073.stdout:2995 read 0x84f654d thru 0x8503fec (0xdaa0 bytes) 2024-09-21T14:21:59.880 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.880 INFO:teuthology.orchestra.run.smithi073.stdout:2997 write 0x812524b thru 0x812de12 (0x8bc8 bytes) 2024-09-21T14:21:59.883 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.883 INFO:teuthology.orchestra.run.smithi073.stdout:2999 read 0x18c464d thru 0x18d0034 (0xb9e8 bytes) 2024-09-21T14:21:59.883 INFO:teuthology.orchestra.run.smithi073.stdout:3000 read 0x7120ecb thru 0x712c9a8 (0xbade bytes) 2024-09-21T14:21:59.883 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.887 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:21:59.888 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:21:59.889 INFO:teuthology.orchestra.run.smithi073.stdout:3004 writesame 0xcfce061 thru 0xcfd2812 data_size 0x72(0x47b2 bytes) 2024-09-21T14:21:59.891 INFO:teuthology.orchestra.run.smithi073.stdout:3005 punch from 0xf4eabd to 0xf53df5, (0x5338 bytes) 2024-09-21T14:21:59.891 INFO:teuthology.orchestra.run.smithi073.stdout:3006 read 0x8c45a4b thru 0x8c4bf6b (0x6521 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3007 read 0xb41a988 thru 0xb428737 (0xddb0 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3008 punch from 0xb7957d2 to 0xb7a54a6, (0xfcd4 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3009 punch from 0xa5c60 to 0xaf409, (0x97a9 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3010 punch from 0x1aa9260 to 0x1ab3450, (0xa1f0 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3012 read 0x5675b2f thru 0x567e1ce (0x86a0 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3013 punch from 0x56ca2ce to 0x56ce3e5, (0x4117 bytes) 2024-09-21T14:21:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:3014 writesame 0xb7811bc thru 0xb78ffbb data_size 0x1c(0xee00 bytes) 2024-09-21T14:21:59.895 INFO:teuthology.orchestra.run.smithi073.stdout:3015 read 0x5980114 thru 0x5988bb8 (0x8aa5 bytes) 2024-09-21T14:21:59.897 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:21:59.905 INFO:teuthology.orchestra.run.smithi073.stdout:3016 trunc from 0xcfd2813 to 0x438c5ff 2024-09-21T14:21:59.924 INFO:teuthology.orchestra.run.smithi073.stdout:3017 read 0x16bc9d thru 0x178eb1 (0xd215 bytes) 2024-09-21T14:21:59.925 INFO:teuthology.orchestra.run.smithi073.stdout:3019 read 0x3e34a77 thru 0x3e40324 (0xb8ae bytes) 2024-09-21T14:21:59.925 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.925 INFO:teuthology.orchestra.run.smithi073.stdout:3022 punch from 0x31bd842 to 0x31c3a5e, (0x621c bytes) 2024-09-21T14:21:59.925 INFO:teuthology.orchestra.run.smithi073.stdout:3023 read 0x42601d0 thru 0x426c837 (0xc668 bytes) 2024-09-21T14:21:59.937 INFO:teuthology.orchestra.run.smithi073.stdout:3024 trunc from 0x438c5ff to 0xd5a382c 2024-09-21T14:21:59.944 INFO:teuthology.orchestra.run.smithi073.stdout:3025 write 0xc8791f5 thru 0xc887cc5 (0xead1 bytes) 2024-09-21T14:21:59.949 INFO:teuthology.orchestra.run.smithi073.stdout:3026 read 0x1c69263 thru 0x1c71e95 (0x8c33 bytes) 2024-09-21T14:21:59.950 INFO:teuthology.orchestra.run.smithi073.stdout:3027 write 0x104588b thru 0x104e3b2 (0x8b28 bytes) 2024-09-21T14:21:59.952 INFO:teuthology.orchestra.run.smithi073.stdout:3028 write 0x497d9b9 thru 0x4988751 (0xad99 bytes) 2024-09-21T14:21:59.954 INFO:teuthology.orchestra.run.smithi073.stdout:3031 punch from 0x3b2fb30 to 0x3b37cec, (0x81bc bytes) 2024-09-21T14:21:59.954 INFO:teuthology.orchestra.run.smithi073.stdout:3033 write 0xa09be99 thru 0xa09e661 (0x27c9 bytes) 2024-09-21T14:21:59.957 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.957 INFO:teuthology.orchestra.run.smithi073.stdout:3036 write 0x5866925 thru 0x586d9e6 (0x70c2 bytes) 2024-09-21T14:21:59.959 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.959 INFO:teuthology.orchestra.run.smithi073.stdout:3039 read 0xf294b5 thru 0xf2ecc1 (0x580d bytes) 2024-09-21T14:21:59.960 INFO:teuthology.orchestra.run.smithi073.stdout:3042 write 0xb8e07c thru 0xb8f080 (0x1005 bytes) 2024-09-21T14:21:59.961 INFO:teuthology.orchestra.run.smithi073.stdout:3044 write 0x635cdce thru 0x6360814 (0x3a47 bytes) 2024-09-21T14:21:59.963 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.963 INFO:teuthology.orchestra.run.smithi073.stdout:3049 punch from 0x232663e to 0x2330e1c, (0xa7de bytes) 2024-09-21T14:21:59.963 INFO:teuthology.orchestra.run.smithi073.stdout:3051 write 0xc1c6111 thru 0xc1ccc2f (0x6b1f bytes) 2024-09-21T14:21:59.964 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:21:59.964+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:21:59.966 INFO:teuthology.orchestra.run.smithi073.stdout:3052 read 0x96e6efe thru 0x96eceb9 (0x5fbc bytes) 2024-09-21T14:21:59.966 INFO:teuthology.orchestra.run.smithi073.stdout:3053 read 0x8ef7112 thru 0x8ef8c7c (0x1b6b bytes) 2024-09-21T14:21:59.966 INFO:teuthology.orchestra.run.smithi073.stdout:3055 punch from 0xcd652b9 to 0xcd70222, (0xaf69 bytes) 2024-09-21T14:21:59.966 INFO:teuthology.orchestra.run.smithi073.stdout:3056 writesame 0xadf6795 thru 0xae063b4 data_size 0x10(0xfc20 bytes) 2024-09-21T14:21:59.970 INFO:teuthology.orchestra.run.smithi073.stdout:3057 writesame 0xcfba438 thru 0xcfbd85e data_size 0xd(0x3427 bytes) 2024-09-21T14:21:59.971 INFO:teuthology.orchestra.run.smithi073.stdout:3058 punch from 0xc7323e5 to 0xc73dfd6, (0xbbf1 bytes) 2024-09-21T14:21:59.971 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.971 INFO:teuthology.orchestra.run.smithi073.stdout:3061 writesame 0x3c222de thru 0x3c289a2 data_size 0x1(0x66c5 bytes) 2024-09-21T14:21:59.991 INFO:teuthology.orchestra.run.smithi073.stdout:3062 write 0xe2420a8 thru 0xe251bcb (0xfb24 bytes) 2024-09-21T14:21:59.993 INFO:teuthology.orchestra.run.smithi073.stdout:3063 read 0x41afc27 thru 0x41b7e05 (0x81df bytes) 2024-09-21T14:21:59.993 INFO:teuthology.orchestra.run.smithi073.stdout:3064 punch from 0x379546d to 0x379d8d7, (0x846a bytes) 2024-09-21T14:21:59.993 INFO:teuthology.orchestra.run.smithi073.stdout:3068 write 0xcc68781 thru 0xcc688f9 (0x179 bytes) 2024-09-21T14:21:59.995 INFO:teuthology.orchestra.run.smithi073.stdout:3070 punch from 0xe0e5a9 to 0xe184f3, (0x9f4a bytes) 2024-09-21T14:21:59.995 INFO:teuthology.orchestra.run.smithi073.stdout:3071 write 0x396fed2 thru 0x397ae15 (0xaf44 bytes) 2024-09-21T14:21:59.996 INFO:teuthology.orchestra.run.smithi073.stdout:3072 read 0x3b4805 thru 0x3c3e4e (0xf64a bytes) 2024-09-21T14:21:59.997 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:21:59.997 INFO:teuthology.orchestra.run.smithi073.stdout:3074 write 0x2fb6f30 thru 0x2fb7ca8 (0xd79 bytes) 2024-09-21T14:22:00.005 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:00.005 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:00.007 INFO:teuthology.orchestra.run.smithi073.stdout:3075 trunc from 0xe251bcc to 0x650d173 2024-09-21T14:22:00.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:00.021 INFO:teuthology.orchestra.run.smithi073.stdout:3076 read 0x99eef5 thru 0x9a3c34 (0x4d40 bytes) 2024-09-21T14:22:00.032 INFO:teuthology.orchestra.run.smithi073.stdout:3077 write 0x999d6b9 thru 0x999e4c8 (0xe10 bytes) 2024-09-21T14:22:00.042 INFO:teuthology.orchestra.run.smithi073.stdout:3079 write 0xb754ad5 thru 0xb763aed (0xf019 bytes) 2024-09-21T14:22:00.051 INFO:teuthology.orchestra.run.smithi073.stdout:3080 trunc from 0xb763aee to 0x6b01c8c 2024-09-21T14:22:00.063 INFO:teuthology.orchestra.run.smithi073.stdout:3081 punch from 0x192338d to 0x1931a1d, (0xe690 bytes) 2024-09-21T14:22:00.064 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:00.064+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:00.079 INFO:teuthology.orchestra.run.smithi073.stdout:3083 writesame 0xdd36417 thru 0xdd3fe04 data_size 0x3d(0x99ee bytes) 2024-09-21T14:22:00.079 INFO:teuthology.orchestra.run.smithi073.stdout:3084 writesame 0xb6dd66f thru 0xb6e3d74 data_size 0x1(0x6706 bytes) 2024-09-21T14:22:00.092 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.092 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.094 INFO:teuthology.orchestra.run.smithi073.stdout:3088 trunc from 0xdd3fe05 to 0xbbd1c33 2024-09-21T14:22:00.103 INFO:teuthology.orchestra.run.smithi073.stdout:3089 punch from 0x3291f4c to 0x329dac3, (0xbb77 bytes) 2024-09-21T14:22:00.103 INFO:teuthology.orchestra.run.smithi073.stdout:3090 read 0x75b2919 thru 0x75bcd6d (0xa455 bytes) 2024-09-21T14:22:00.103 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.103 INFO:teuthology.orchestra.run.smithi073.stdout:3092 write 0x47e9a9d thru 0x47ec60b (0x2b6f bytes) 2024-09-21T14:22:00.104 INFO:teuthology.orchestra.run.smithi073.stdout:3093 read 0x31818ca thru 0x318f7f4 (0xdf2b bytes) 2024-09-21T14:22:00.105 INFO:teuthology.orchestra.run.smithi073.stdout:3094 punch from 0xb9f7ff to 0xbae63f, (0xee40 bytes) 2024-09-21T14:22:00.105 INFO:teuthology.orchestra.run.smithi073.stdout:3097 read 0x6a56a81 thru 0x6a63124 (0xc6a4 bytes) 2024-09-21T14:22:00.105 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.105 INFO:teuthology.orchestra.run.smithi073.stdout:3099 punch from 0x441a2c0 to 0x442477d, (0xa4bd bytes) 2024-09-21T14:22:00.117 INFO:teuthology.orchestra.run.smithi073.stdout:3100 trunc from 0xbbd1c33 to 0x1362e11 2024-09-21T14:22:00.148 INFO:teuthology.orchestra.run.smithi073.stdout:3101 write 0xa95cc2f thru 0xa96221c (0x55ee bytes) 2024-09-21T14:22:00.151 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.151 INFO:teuthology.orchestra.run.smithi073.stdout:3103 writesame 0xa09bd47 thru 0xa09d85b data_size 0x1(0x1b15 bytes) 2024-09-21T14:22:00.155 INFO:teuthology.orchestra.run.smithi073.stdout:3104 punch from 0x8783083 to 0x8792c45, (0xfbc2 bytes) 2024-09-21T14:22:00.156 INFO:teuthology.orchestra.run.smithi073.stdout:3105 read 0xda3de7 thru 0xdb0ec8 (0xd0e2 bytes) 2024-09-21T14:22:00.161 INFO:teuthology.orchestra.run.smithi073.stdout:3106 write 0xaee8b3c thru 0xaeef599 (0x6a5e bytes) 2024-09-21T14:22:00.162 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:00.162 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:00.164 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:00.165+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:00.165 INFO:teuthology.orchestra.run.smithi073.stdout:3107 write 0x40bdb27 thru 0x40cae7b (0xd355 bytes) 2024-09-21T14:22:00.166 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.166 INFO:teuthology.orchestra.run.smithi073.stdout:3110 read 0x338d014 thru 0x3396e85 (0x9e72 bytes) 2024-09-21T14:22:00.167 INFO:teuthology.orchestra.run.smithi073.stdout:3111 write 0x8f02f1d thru 0x8f109bb (0xda9f bytes) 2024-09-21T14:22:00.168 INFO:teuthology.orchestra.run.smithi073.stdout:3113 writesame 0x17d3889 thru 0x17d8936 data_size 0x17(0x50ae bytes) 2024-09-21T14:22:00.169 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.170 INFO:teuthology.orchestra.run.smithi073.stdout:3116 punch from 0x11707a8 to 0x117837e, (0x7bd6 bytes) 2024-09-21T14:22:00.170 INFO:teuthology.orchestra.run.smithi073.stdout:3117 trunc from 0xaeef59a to 0xaf113e7 2024-09-21T14:22:00.172 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:00.174 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.174 INFO:teuthology.orchestra.run.smithi073.stdout:3120 trunc from 0xaf113e7 to 0xb09dc6a 2024-09-21T14:22:00.179 INFO:teuthology.orchestra.run.smithi073.stdout:3121 write 0x6c0d317 thru 0x6c1c3b4 (0xf09e bytes) 2024-09-21T14:22:00.180 INFO:teuthology.orchestra.run.smithi073.stdout:3122 read 0x8a5c0cd thru 0x8a5f4bd (0x33f1 bytes) 2024-09-21T14:22:00.180 INFO:teuthology.orchestra.run.smithi073.stdout:3123 punch from 0x912eeac to 0x913ee21, (0xff75 bytes) 2024-09-21T14:22:00.180 INFO:teuthology.orchestra.run.smithi073.stdout:3124 read 0x777fef4 thru 0x7781024 (0x1131 bytes) 2024-09-21T14:22:00.180 INFO:teuthology.orchestra.run.smithi073.stdout:3126 read 0x97345f0 thru 0x9737e35 (0x3846 bytes) 2024-09-21T14:22:00.181 INFO:teuthology.orchestra.run.smithi073.stdout:3127 write 0x9b78154 thru 0x9b7ffa5 (0x7e52 bytes) 2024-09-21T14:22:00.183 INFO:teuthology.orchestra.run.smithi073.stdout:3128 punch from 0x5279162 to 0x5281d08, (0x8ba6 bytes) 2024-09-21T14:22:00.193 INFO:teuthology.orchestra.run.smithi073.stdout:3129 trunc from 0xb09dc6a to 0x2a4be38 2024-09-21T14:22:00.212 INFO:teuthology.orchestra.run.smithi073.stdout:3130 write 0x41891ae thru 0x418dc69 (0x4abc bytes) 2024-09-21T14:22:00.221 INFO:teuthology.orchestra.run.smithi073.stdout:3131 write 0x5ebca57 thru 0x5ec7337 (0xa8e1 bytes) 2024-09-21T14:22:00.222 INFO:teuthology.orchestra.run.smithi073.stdout:3132 read 0x34d7d65 thru 0x34e0ff3 (0x928f bytes) 2024-09-21T14:22:00.222 INFO:teuthology.orchestra.run.smithi073.stdout:3133 read 0x1dcdbe1 thru 0x1dcf9f5 (0x1e15 bytes) 2024-09-21T14:22:00.223 INFO:teuthology.orchestra.run.smithi073.stdout:3134 punch from 0x2ffa8cc to 0x2ffb61d, (0xd51 bytes) 2024-09-21T14:22:00.223 INFO:teuthology.orchestra.run.smithi073.stdout:3136 read 0x3a8f627 thru 0x3a9c9e8 (0xd3c2 bytes) 2024-09-21T14:22:00.223 INFO:teuthology.orchestra.run.smithi073.stdout:3137 punch from 0x331a0e2 to 0x331efae, (0x4ecc bytes) 2024-09-21T14:22:00.226 INFO:teuthology.orchestra.run.smithi073.stdout:3138 trunc from 0x5ec7338 to 0x37df030 2024-09-21T14:22:00.234 INFO:teuthology.orchestra.run.smithi073.stdout:3139 read 0x1211cde thru 0x1214335 (0x2658 bytes) 2024-09-21T14:22:00.234 INFO:teuthology.orchestra.run.smithi073.stdout:3140 read 0x295e34a thru 0x2964640 (0x62f7 bytes) 2024-09-21T14:22:00.249 INFO:teuthology.orchestra.run.smithi073.stdout:3141 write 0xafc660c thru 0xafce65e (0x8053 bytes) 2024-09-21T14:22:00.252 INFO:teuthology.orchestra.run.smithi073.stdout:3142 write 0x969bad6 thru 0x96a84ba (0xc9e5 bytes) 2024-09-21T14:22:00.254 INFO:teuthology.orchestra.run.smithi073.stdout:3143 read 0x9df3c73 thru 0x9df8ee7 (0x5275 bytes) 2024-09-21T14:22:00.259 INFO:teuthology.orchestra.run.smithi073.stdout:3144 trunc from 0xafce65f to 0x6fd0558 2024-09-21T14:22:00.265 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:00.266+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:00.283 INFO:teuthology.orchestra.run.smithi073.stdout:3145 writesame 0xbd45d23 thru 0xbd4d68a data_size 0x3c(0x7968 bytes) 2024-09-21T14:22:00.283 INFO:teuthology.orchestra.run.smithi073.stdout:3146 punch from 0xa1f27d to 0xa2e3fb, (0xf17e bytes) 2024-09-21T14:22:00.283 INFO:teuthology.orchestra.run.smithi073.stdout:3148 write 0x3a94033 thru 0x3aa12ad (0xd27b bytes) 2024-09-21T14:22:00.287 INFO:teuthology.orchestra.run.smithi073.stdout:3149 trunc from 0xbd4d68b to 0x987fc9f 2024-09-21T14:22:00.299 INFO:teuthology.orchestra.run.smithi073.stdout:3150 write 0xa094640 thru 0xa0a0a8a (0xc44b bytes) 2024-09-21T14:22:00.302 INFO:teuthology.orchestra.run.smithi073.stdout:3152 punch from 0x498d9a8 to 0x498ffaa, (0x2602 bytes) 2024-09-21T14:22:00.313 INFO:teuthology.orchestra.run.smithi073.stdout:3153 write 0xe6363ce thru 0xe64413f (0xdd72 bytes) 2024-09-21T14:22:00.320 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.320 INFO:teuthology.orchestra.run.smithi073.stdout:3155 write 0x5aae9f2 thru 0x5ab5b1a (0x7129 bytes) 2024-09-21T14:22:00.321 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:00.321 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:00.322 INFO:teuthology.orchestra.run.smithi073.stdout:3156 writesame 0xa6e2bf thru 0xa7d780 data_size 0x12(0xf4c2 bytes) 2024-09-21T14:22:00.324 INFO:teuthology.orchestra.run.smithi073.stdout:3158 read 0x194577f thru 0x19548cd (0xf14f bytes) 2024-09-21T14:22:00.325 INFO:teuthology.orchestra.run.smithi073.stdout:3159 punch from 0xb012541 to 0xb018341, (0x5e00 bytes) 2024-09-21T14:22:00.325 INFO:teuthology.orchestra.run.smithi073.stdout:3160 clone 27 order 25 su 8388608 sc 5 2024-09-21T14:22:00.331 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:00.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:00.386 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:00.386+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:00.393 INFO:teuthology.orchestra.run.smithi073.stderr:set 3.d pg_upmap mapping to [6,2] 2024-09-21T14:22:00.466 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:00.467+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:00.469 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:00.470 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:00.479 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:00.480 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:00.487 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:00.487 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:00.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:00.495 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone26 from 0xe644140 (overlap 0x10f0f81) to 0x6c6c9a 2024-09-21T14:22:00.504 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:00.504 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:00.526 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:00.526 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:00.543 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:00.560 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:00.560 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:00.567 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:00.567+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:00.576 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:00.577 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:00.593 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:00.593 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:00.607 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T14:22:00.608 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.622 INFO:teuthology.orchestra.run.smithi073.stdout:3162 trunc from 0xe644140 to 0x2ec3106 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207789115000001, 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207772472999999, 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207747423000001, 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207729271000002, 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207706034000001, 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207684966999999, 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:00.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:00.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:00.665 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207668606999999, 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:00.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:00.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.669 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207638327000002, 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:00.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:00.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:00.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207623189, 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.674 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:00.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:00.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207603593000002, 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:00.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:00.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:00.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:00.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207594816, 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:00.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207582455000001, 2024-09-21T14:22:00.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:00.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:00.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:00.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207573038, 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.693 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207551847000001, 2024-09-21T14:22:00.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207534739, 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207517796000001, 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207501417, 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:00.715 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207460056999999, 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:00.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:00.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.207440376000001, 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.720 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:00.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 7.8025519289999998, 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:00.724 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:00.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:00.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:00.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:00.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:00.729 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:00.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3163 punch from 0x613764 to 0x614e88, (0x1724 bytes) 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3166 read 0x1889675 thru 0x18907ce (0x715a bytes) 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3167 writesame 0x8a42003 thru 0x8a4aeb3 data_size 0x1(0x8eb1 bytes) 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3170 write 0x5f321b8 thru 0x5f3b190 (0x8fd9 bytes) 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3171 read 0x2322130 thru 0x232260d (0x4de bytes) 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3172 trunc from 0x8a4aeb4 to 0x852cb9 2024-09-21T14:22:00.732 INFO:teuthology.orchestra.run.smithi073.stdout:3173 read 0x3a8b86 thru 0x3b5964 (0xcddf bytes) 2024-09-21T14:22:00.733 INFO:teuthology.orchestra.run.smithi073.stdout:3175 write 0xd6c08a4 thru 0xd6cd6c4 (0xce21 bytes) 2024-09-21T14:22:00.733 INFO:teuthology.orchestra.run.smithi073.stdout:3176 read 0xb49aa47 thru 0xb4a15d8 (0x6b92 bytes) 2024-09-21T14:22:00.733 INFO:teuthology.orchestra.run.smithi073.stdout:3177 trunc from 0xd6cd6c5 to 0x4e3caac 2024-09-21T14:22:00.733 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:00.730+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:00.742 INFO:teuthology.orchestra.run.smithi073.stdout:3178 trunc from 0x4e3caac to 0x66cd1c8 2024-09-21T14:22:00.758 INFO:teuthology.orchestra.run.smithi073.stdout:3179 trunc from 0x66cd1c8 to 0xea9693a 2024-09-21T14:22:00.775 INFO:teuthology.orchestra.run.smithi073.stdout:3183 trunc from 0xea9693a to 0x4ee2312 2024-09-21T14:22:00.792 INFO:teuthology.orchestra.run.smithi073.stdout:3184 write 0x809a395 thru 0x80a609d (0xbd09 bytes) 2024-09-21T14:22:00.794 INFO:teuthology.orchestra.run.smithi073.stdout:3186 write 0x4e95590 thru 0x4ea3034 (0xdaa5 bytes) 2024-09-21T14:22:00.806 INFO:teuthology.orchestra.run.smithi073.stdout:3187 write 0xa4c6d72 thru 0xa4cf57c (0x880b bytes) 2024-09-21T14:22:00.809 INFO:teuthology.orchestra.run.smithi073.stdout:3188 punch from 0x6eb2a3c to 0x6eb8998, (0x5f5c bytes) 2024-09-21T14:22:00.809 INFO:teuthology.orchestra.run.smithi073.stdout:3189 writesame 0x76c00c thru 0x7758df data_size 0x2(0x98d4 bytes) 2024-09-21T14:22:00.819 INFO:teuthology.orchestra.run.smithi073.stdout:3190 write 0x5bdd455 thru 0x5be1919 (0x44c5 bytes) 2024-09-21T14:22:00.821 INFO:teuthology.orchestra.run.smithi073.stdout:3192 write 0x92b2c0e thru 0x92b4a9a (0x1e8d bytes) 2024-09-21T14:22:00.823 INFO:teuthology.orchestra.run.smithi073.stdout:3193 writesame 0xa0c3b24 thru 0xa0cb8f0 data_size 0x5f(0x7dcd bytes) 2024-09-21T14:22:00.825 INFO:teuthology.orchestra.run.smithi073.stdout:3195 write 0x72d1e3 thru 0x73588e (0x86ac bytes) 2024-09-21T14:22:00.826 INFO:teuthology.orchestra.run.smithi073.stdout:3196 write 0x6f6d7c8 thru 0x6f7d14d (0xf986 bytes) 2024-09-21T14:22:00.831 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:00.832+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:00.837 INFO:teuthology.orchestra.run.smithi073.stdout:3199 writesame 0xce580e4 thru 0xce64875 data_size 0x83(0xc792 bytes) 2024-09-21T14:22:00.838 INFO:teuthology.orchestra.run.smithi073.stdout:3201 write 0x196b83 thru 0x1a6109 (0xf587 bytes) 2024-09-21T14:22:00.839 INFO:teuthology.orchestra.run.smithi073.stdout:3202 punch from 0x31bf00f to 0x31c9d84, (0xad75 bytes) 2024-09-21T14:22:00.840 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:00.841 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:00.841 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:00.841 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:00.845 INFO:teuthology.orchestra.run.smithi073.stdout:3203 write 0xd37b376 thru 0xd37e267 (0x2ef2 bytes) 2024-09-21T14:22:00.851 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:00.860 INFO:teuthology.orchestra.run.smithi073.stdout:3204 trunc from 0xd37e268 to 0x2d1cc6f 2024-09-21T14:22:00.872 INFO:teuthology.orchestra.run.smithi073.stdout:3205 read 0x2710ae9 thru 0x2717192 (0x66aa bytes) 2024-09-21T14:22:00.872 INFO:teuthology.orchestra.run.smithi073.stdout:3206 read 0x237a86a thru 0x2386e2f (0xc5c6 bytes) 2024-09-21T14:22:00.872 INFO:teuthology.orchestra.run.smithi073.stdout:3207 read 0x273955 thru 0x27cd01 (0x93ad bytes) 2024-09-21T14:22:00.886 INFO:teuthology.orchestra.run.smithi073.stdout:3208 write 0x97ced82 thru 0x97d678d (0x7a0c bytes) 2024-09-21T14:22:00.888 INFO:teuthology.orchestra.run.smithi073.stdout:3209 read 0x346aa7f thru 0x3472aa8 (0x802a bytes) 2024-09-21T14:22:00.888 INFO:teuthology.orchestra.run.smithi073.stdout:3211 writesame 0x3ec9542 thru 0x3ed747c data_size 0x2b(0xdf3b bytes) 2024-09-21T14:22:00.892 INFO:teuthology.orchestra.run.smithi073.stdout:3212 write 0x6c7f656 thru 0x6c882c1 (0x8c6c bytes) 2024-09-21T14:22:00.893 INFO:teuthology.orchestra.run.smithi073.stdout:3213 trunc from 0x97d678e to 0x967d1de 2024-09-21T14:22:00.900 INFO:teuthology.orchestra.run.smithi073.stdout:3215 punch from 0xf486e2 to 0xf4d4a8, (0x4dc6 bytes) 2024-09-21T14:22:00.909 INFO:teuthology.orchestra.run.smithi073.stdout:3217 write 0xc5705e7 thru 0xc5764ef (0x5f09 bytes) 2024-09-21T14:22:00.913 INFO:teuthology.orchestra.run.smithi073.stdout:3218 read 0x9a75ddb thru 0x9a7e3d5 (0x85fb bytes) 2024-09-21T14:22:00.914 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:00.914 INFO:teuthology.orchestra.run.smithi073.stdout:3221 clone 28 order 24 su 262144 sc 5 2024-09-21T14:22:00.932 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:00.932+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:00.994 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:00.994 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:00.994 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:00.994 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:01.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:01.033 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:01.033+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:01.106 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:01.106 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:01.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:01.106 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:01.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 36.867460346999998, 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509478983000001, 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:22:01.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509461656999999, 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.115 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:01.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:01.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509444143, 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:01.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509426889, 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.123 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509399039000002, 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509377435000001, 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509360437000002, 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509343417, 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509325966999999, 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509308828999998, 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509285601999999, 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509269327999998, 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509244956, 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509222389000001, 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509205468000001, 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509188969, 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509167972, 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.509145914000001, 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.3350750419999997, 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:01.194 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:01.195 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:01.195+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:01.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:01.296 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:01.296+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:01.299 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:01.299 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:01.299 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:01.299 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:01.307 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:01.397 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:01.397+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:01.413 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:01.413 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:01.413 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:01.413 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:01.413 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:01.422 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:01.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:22:01.506 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:01.506+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:01.508 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone27 from 0xc5764f0 (overlap 0x852cb9) to 0x27f3d1 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43037 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:01.577 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193209724000001, 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044045162999999998, 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43037 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382704+0000", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1880000000000001e-06 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382710+0000", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.507e-06 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382714+0000", 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2030000000000002e-06 2024-09-21T14:22:01.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426582+0000", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043868468000000001 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426657+0000", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4577000000000001e-05 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426748+0000", 2024-09-21T14:22:01.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.056e-05 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43038 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 319488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193193840999999, 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044219614999999997, 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43038 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382724+0000", 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5659999999999997e-06 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382729+0000", 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382733+0000", 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426770+0000", 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044036868 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426847+0000", 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7068000000000001e-05 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426938+0000", 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0708e-05 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.586 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43039 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193147726999999, 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044367363999999999, 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43039 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382767+0000", 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5959999999999998e-06 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382773+0000", 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6710000000000002e-06 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382779+0000", 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426961+0000", 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044182402000000003 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427041+0000", 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9810999999999999e-05 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427132+0000", 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1025000000000003e-05 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43040 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.19312948, 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044540541000000003, 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43040 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382782+0000", 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382784+0000", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0209999999999998e-06 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382790+0000", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5130000000000004e-06 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382794+0000", 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2200000000000003e-06 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427155+0000", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044360978000000002 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427234+0000", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8580999999999999e-05 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427323+0000", 2024-09-21T14:22:01.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9584000000000002e-05 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43041 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 354304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193114217, 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044711682000000003, 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43041 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382799+0000", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382805+0000", 2024-09-21T14:22:01.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7259999999999999e-06 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382810+0000", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2910000000000001e-06 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427346+0000", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044536282000000003 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:22:01.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4651000000000001e-05 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427510+0000", 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9147999999999994e-05 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43042 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 364544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193097290000001, 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044890194000000001, 2024-09-21T14:22:01.600 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43042 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:01.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382814+0000", 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382816+0000", 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9259999999999999e-06 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382822+0000", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.733e-06 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382826+0000", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.929e-06 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427536+0000", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044709567999999998 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427613+0000", 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7390000000000003e-05 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427705+0000", 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1984000000000006e-05 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43043 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:01.604 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193082251, 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045064142000000001, 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43043 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382831+0000", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.874e-06 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382837+0000", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8320000000000002e-06 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382841+0000", 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1500000000000001e-06 2024-09-21T14:22:01.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427728+0000", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044886479 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427805+0000", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.661e-05 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427894+0000", 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9488999999999996e-05 2024-09-21T14:22:01.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43044 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193066898, 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045458112000000002, 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43044 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.609 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:01.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382847+0000", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9810000000000002e-06 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382853+0000", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7880000000000003e-06 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382857+0000", 2024-09-21T14:22:01.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9790000000000004e-06 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427917+0000", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045060351999999998 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428174+0000", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025663200000000002 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428303+0000", 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129701 2024-09-21T14:22:01.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43045 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193051548, 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045669176999999998, 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43045 2024-09-21T14:22:01.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382860+0000", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382862+0000", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382868+0000", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6500000000000001e-06 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382873+0000", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5570000000000003e-06 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428330+0000", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045456929 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428406+0000", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5184000000000004e-05 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428530+0000", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124213 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43046 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193034911, 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873328999999997, 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43046 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382879+0000", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9800000000000001e-06 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382884+0000", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5929999999999996e-06 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382888+0000", 2024-09-21T14:22:01.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9249999999999997e-06 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428559+0000", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045670193999999997 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428632+0000", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2978999999999994e-05 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428751+0000", 2024-09-21T14:22:01.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119015 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43047 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193019734, 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046080838999999998, 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43047 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382894+0000", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.956e-06 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382900+0000", 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.541e-06 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382904+0000", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0230000000000003e-06 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428777+0000", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873239000000003 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428850+0000", 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2762000000000006e-05 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428973+0000", 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012365400000000001 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43048 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.193004795, 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046302055000000002, 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43048 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:01.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382909+0000", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.835e-06 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382927+0000", 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8580000000000002e-05 2024-09-21T14:22:01.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382932+0000", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5419999999999999e-06 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429004+0000", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046071809999999998 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429077+0000", 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3615999999999994e-05 2024-09-21T14:22:01.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429209+0000", 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199699999999999 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43049 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192976423999999, 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492449999999998, 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43049 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:01.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382935+0000", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382937+0000", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382943+0000", 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:01.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382947+0000", 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0670000000000002e-06 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429235+0000", 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046287933000000003 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429308+0000", 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2879000000000005e-05 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429428+0000", 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120269 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43050 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192961154000001, 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046740351999999999, 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43050 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382968+0000", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7289e-05 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383094+0000", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012638999999999999 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383114+0000", 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0081999999999998e-05 2024-09-21T14:22:01.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429454+0000", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046339578999999999 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429527+0000", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3450999999999999e-05 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429691+0000", 2024-09-21T14:22:01.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016386100000000001 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43051 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192731335, 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046543382000000001, 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43051 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383180+0000", 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383184+0000", 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7510000000000002e-06 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383207+0000", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2908000000000002e-05 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383231+0000", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3981999999999998e-05 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429717+0000", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046485897999999998 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429724+0000", 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5039999999999996e-06 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43052 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192647236999999, 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046500398999999998, 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43052 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:01.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383264+0000", 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383281+0000", 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6515e-05 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383287+0000", 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4439999999999996e-06 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383294+0000", 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1680000000000001e-06 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429747+0000", 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046453853000000003 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429765+0000", 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7918999999999999e-05 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43053 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192612947000001, 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492406, 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43053 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383301+0000", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3580000000000001e-06 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383307+0000", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9619999999999998e-06 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383311+0000", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429785+0000", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046474161 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429792+0000", 2024-09-21T14:22:01.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.139e-06 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43054 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192597170000001, 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046502169000000003, 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43054 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383317+0000", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9709999999999998e-06 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383322+0000", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5589999999999996e-06 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383328+0000", 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046481947000000003 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429817+0000", 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.653 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43055 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192580074, 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046513090999999999, 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43055 2024-09-21T14:22:01.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:01.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383337+0000", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383342+0000", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4559999999999999e-06 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383347+0000", 2024-09-21T14:22:01.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3830000000000002e-06 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429839+0000", 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492377000000001 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429845+0000", 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1689999999999997e-06 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43056 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.192561722000001, 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046522758999999997, 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43056 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:01.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383352+0000", 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383357+0000", 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5380000000000002e-06 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383362+0000", 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2629999999999997e-06 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429865+0000", 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046503567000000003 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429873+0000", 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.858e-06 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:01.661 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:01.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:01.663 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:01.662+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:01.670 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T14:22:01.683 INFO:teuthology.orchestra.run.smithi073.stdout:3222 read 0x380e28a thru 0x380e775 (0x4ec bytes) 2024-09-21T14:22:01.685 INFO:teuthology.orchestra.run.smithi073.stdout:3223 write 0x7155f23 thru 0x7161e89 (0xbf67 bytes) 2024-09-21T14:22:01.697 INFO:teuthology.orchestra.run.smithi073.stdout:3224 writesame 0x16d9bf1 thru 0x16dbfeb data_size 0x1(0x23fb bytes) 2024-09-21T14:22:01.710 INFO:teuthology.orchestra.run.smithi073.stdout:3225 writesame 0x204176a thru 0x20485bc data_size 0x1(0x6e53 bytes) 2024-09-21T14:22:01.729 INFO:teuthology.orchestra.run.smithi073.stdout:3226 write 0x91a4c16 thru 0x91ace82 (0x826d bytes) 2024-09-21T14:22:01.736 INFO:teuthology.orchestra.run.smithi073.stdout:3228 read 0x59771ab thru 0x597c4ca (0x5320 bytes) 2024-09-21T14:22:01.737 INFO:teuthology.orchestra.run.smithi073.stdout:3231 write 0x4f944ec thru 0x4f9c985 (0x849a bytes) 2024-09-21T14:22:01.746 INFO:teuthology.orchestra.run.smithi073.stdout:3235 read 0x1e104ec thru 0x1e13125 (0x2c3a bytes) 2024-09-21T14:22:01.747 INFO:teuthology.orchestra.run.smithi073.stdout:3236 write 0x307a69b thru 0x3086dfc (0xc762 bytes) 2024-09-21T14:22:01.749 INFO:teuthology.orchestra.run.smithi073.stdout:3237 read 0x847957e thru 0x8485512 (0xbf95 bytes) 2024-09-21T14:22:01.749 INFO:teuthology.orchestra.run.smithi073.stdout:3239 read 0x35e17d4 thru 0x35ec88e (0xb0bb bytes) 2024-09-21T14:22:01.754 INFO:teuthology.orchestra.run.smithi073.stdout:3241 trunc from 0xc5764f0 to 0x977fa76 2024-09-21T14:22:01.763 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:01.763+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:01.775 INFO:teuthology.orchestra.run.smithi073.stdout:3244 write 0x286f60a thru 0x2875988 (0x637f bytes) 2024-09-21T14:22:01.777 INFO:teuthology.orchestra.run.smithi073.stdout:3246 write 0x831881f thru 0x83232ac (0xaa8e bytes) 2024-09-21T14:22:01.778 INFO:teuthology.orchestra.run.smithi073.stdout:3247 writesame 0x7535f70 thru 0x7544cb3 data_size 0xa(0xed44 bytes) 2024-09-21T14:22:01.782 INFO:teuthology.orchestra.run.smithi073.stdout:3249 flatten 2024-09-21T14:22:01.795 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:01.795 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:01.795 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:01.795 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:01.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:01.815 INFO:teuthology.orchestra.run.smithi073.stdout:3250 write 0xca3877e thru 0xca40e78 (0x86fb bytes) 2024-09-21T14:22:01.818 INFO:teuthology.orchestra.run.smithi073.stdout:3251 write 0x8fd6529 thru 0x8fd6872 (0x34a bytes) 2024-09-21T14:22:01.827 INFO:teuthology.orchestra.run.smithi073.stdout:3252 trunc from 0xca40e79 to 0x55c437a 2024-09-21T14:22:01.838 INFO:teuthology.orchestra.run.smithi073.stdout:3254 punch from 0x4c752e5 to 0x4c833f1, (0xe10c bytes) 2024-09-21T14:22:01.839 INFO:teuthology.orchestra.run.smithi073.stdout:3255 read 0x35a7e10 thru 0x35b4602 (0xc7f3 bytes) 2024-09-21T14:22:01.839 INFO:teuthology.orchestra.run.smithi073.stdout:3256 read 0x2a074f thru 0x2b0475 (0xfd27 bytes) 2024-09-21T14:22:01.839 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:01.839 INFO:teuthology.orchestra.run.smithi073.stdout:3261 write 0x8a20b thru 0x8f8af (0x56a5 bytes) 2024-09-21T14:22:01.840 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:01.841 INFO:teuthology.orchestra.run.smithi073.stdout:3263 punch from 0x1556d92 to 0x156688a, (0xfaf8 bytes) 2024-09-21T14:22:01.843 INFO:teuthology.orchestra.run.smithi073.stdout:3264 trunc from 0x55c437a to 0x37e66a1 2024-09-21T14:22:01.864 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:01.864+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:01.870 INFO:teuthology.orchestra.run.smithi073.stdout:3265 write 0xc8c93e9 thru 0xc8d401d (0xac35 bytes) 2024-09-21T14:22:01.879 INFO:teuthology.orchestra.run.smithi073.stdout:3266 writesame 0xdfb9a24 thru 0xdfc11c4 data_size 0x7d(0x77a1 bytes) 2024-09-21T14:22:01.886 INFO:teuthology.orchestra.run.smithi073.stdout:3267 trunc from 0xdfc11c5 to 0x8229ae0 2024-09-21T14:22:01.896 INFO:teuthology.orchestra.run.smithi073.stdout:3268 read 0x7c1e549 thru 0x7c21f12 (0x39ca bytes) 2024-09-21T14:22:01.897 INFO:teuthology.orchestra.run.smithi073.stdout:3269 read 0x2ed7581 thru 0x2ed9698 (0x2118 bytes) 2024-09-21T14:22:01.897 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:01.903 INFO:teuthology.orchestra.run.smithi073.stdout:3273 trunc from 0x8229ae0 to 0xd9cc8f5 2024-09-21T14:22:01.909 INFO:teuthology.orchestra.run.smithi073.stdout:3274 write 0xb495c7f thru 0xb4a006e (0xa3f0 bytes) 2024-09-21T14:22:01.912 INFO:teuthology.orchestra.run.smithi073.stdout:3275 write 0xb22f14a thru 0xb237c15 (0x8acc bytes) 2024-09-21T14:22:01.914 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:01.914 INFO:teuthology.orchestra.run.smithi073.stdout:3277 write 0x80f384b thru 0x80f555e (0x1d14 bytes) 2024-09-21T14:22:01.921 INFO:teuthology.orchestra.run.smithi073.stdout:3278 trunc from 0xd9cc8f5 to 0x9227105 2024-09-21T14:22:01.931 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:01.931 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:01.931 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:01.931 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:01.931 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:01.940 INFO:teuthology.orchestra.run.smithi073.stdout:3279 write 0xb2662ec thru 0xb27608d (0xfda2 bytes) 2024-09-21T14:22:01.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:01.943 INFO:teuthology.orchestra.run.smithi073.stdout:3280 punch from 0x9cba6bd to 0x9cca42d, (0xfd70 bytes) 2024-09-21T14:22:01.943 INFO:teuthology.orchestra.run.smithi073.stdout:3281 read 0x45a3783 thru 0x45a8656 (0x4ed4 bytes) 2024-09-21T14:22:01.943 INFO:teuthology.orchestra.run.smithi073.stdout:3282 writesame 0x1cb9336 thru 0x1cc4996 data_size 0x3(0xb661 bytes) 2024-09-21T14:22:01.949 INFO:teuthology.orchestra.run.smithi073.stdout:3284 read 0x140c69f thru 0x140f24c (0x2bae bytes) 2024-09-21T14:22:01.949 INFO:teuthology.orchestra.run.smithi073.stdout:3285 read 0xa64f7d thru 0xa6bc0c (0x6c90 bytes) 2024-09-21T14:22:01.950 INFO:teuthology.orchestra.run.smithi073.stdout:3286 write 0x5b2e317 thru 0x5b2ebaf (0x899 bytes) 2024-09-21T14:22:01.962 INFO:teuthology.orchestra.run.smithi073.stdout:3287 writesame 0xee3f98f thru 0xee4bcf2 data_size 0x3d(0xc364 bytes) 2024-09-21T14:22:01.963 INFO:teuthology.orchestra.run.smithi073.stdout:3289 read 0x3070575 thru 0x30780f8 (0x7b84 bytes) 2024-09-21T14:22:01.965 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:01.965+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:01.976 INFO:teuthology.orchestra.run.smithi073.stdout:3290 trunc from 0xee4bcf3 to 0x46e5f39 2024-09-21T14:22:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:3291 punch from 0x3e75400 to 0x3e852db, (0xfedb bytes) 2024-09-21T14:22:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:3292 read 0x227a7aa thru 0x228642a (0xbc81 bytes) 2024-09-21T14:22:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:3293 read 0x32e0af6 thru 0x32e4f92 (0x449d bytes) 2024-09-21T14:22:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:3295 read 0x6f50d8 thru 0x6fa39d (0x52c6 bytes) 2024-09-21T14:22:02.002 INFO:teuthology.orchestra.run.smithi073.stdout:3296 writesame 0xcb05001 thru 0xcb0513f data_size 0x1(0x13f bytes) 2024-09-21T14:22:02.002 INFO:teuthology.orchestra.run.smithi073.stdout:3297 write 0x8ae826b thru 0x8af236e (0xa104 bytes) 2024-09-21T14:22:02.005 INFO:teuthology.orchestra.run.smithi073.stdout:3298 read 0xa0af2ce thru 0xa0b58d9 (0x660c bytes) 2024-09-21T14:22:02.005 INFO:teuthology.orchestra.run.smithi073.stdout:3299 read 0x829cc1f thru 0x82a30d8 (0x64ba bytes) 2024-09-21T14:22:02.005 INFO:teuthology.orchestra.run.smithi073.stdout:3300 read 0x1239247 thru 0x124172b (0x84e5 bytes) 2024-09-21T14:22:02.005 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.013 INFO:teuthology.orchestra.run.smithi073.stdout:3302 trunc from 0xcb05140 to 0x5e357ef 2024-09-21T14:22:02.020 INFO:teuthology.orchestra.run.smithi073.stdout:3303 read 0x2655dd4 thru 0x26635af (0xd7dc bytes) 2024-09-21T14:22:02.027 INFO:teuthology.orchestra.run.smithi073.stdout:3304 writesame 0x7b428bc thru 0x7b48e65 data_size 0xe(0x65aa bytes) 2024-09-21T14:22:02.036 INFO:teuthology.orchestra.run.smithi073.stdout:3305 write 0xafbce15 thru 0xafc7cf6 (0xaee2 bytes) 2024-09-21T14:22:02.039 INFO:teuthology.orchestra.run.smithi073.stdout:3307 writesame 0x3959f67 thru 0x395dd93 data_size 0x1(0x3e2d bytes) 2024-09-21T14:22:02.047 INFO:teuthology.orchestra.run.smithi073.stdout:3308 read 0x5a4c207 thru 0x5a52131 (0x5f2b bytes) 2024-09-21T14:22:02.053 INFO:teuthology.orchestra.run.smithi073.stdout:3310 trunc from 0xafc7cf7 to 0x5a8069e 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.840511165999999, 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:02.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:02.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:02.066 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42293 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3274752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.998096167, 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032075181000000001, 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42293 2024-09-21T14:22:02.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063351+0000", 2024-09-21T14:22:02.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063354+0000", 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0880000000000002e-06 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063359+0000", 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7620000000000001e-06 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063364+0000", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3560000000000003e-06 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095339+0000", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031974925000000001 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095389+0000", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0095000000000001e-05 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095427+0000", 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8389000000000001e-05 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42294 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3500032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.998080562, 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032160279, 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.071 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42294 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063369+0000", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0320000000000002e-06 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063375+0000", 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4729999999999999e-06 2024-09-21T14:22:02.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063379+0000", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5329999999999996e-06 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095440+0000", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032060937999999997 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095489+0000", 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8711000000000002e-05 2024-09-21T14:22:02.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095528+0000", 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8926e-05 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53665 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3487201090000003, 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032083390000000003, 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.075 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53665 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712733+0000", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712740+0000", 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9290000000000001e-06 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712746+0000", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744674+0000", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031927981000000001 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744749+0000", 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744811+0000", 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2681000000000005e-05 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3486867040000003, 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:02.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.083 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3486690000000001, 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3486533290000002, 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.088 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:02.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:02.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:02.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.348637815, 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:02.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:02.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:02.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:02.095 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3486227480000004, 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.096 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.099 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3486075529999999, 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:02.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3485924469999997, 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:02.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.348577337, 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3485619719999997, 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3485465129999996, 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.116 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3485314800000001, 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.120 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:02.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.348515591, 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:02.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:02.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:02.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.348489915, 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:02.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3484746300000001, 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:02.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:02.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.348452666, 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:02.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:02.140 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.3484366830000001, 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:02.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:02.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.145 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:02.145 INFO:teuthology.orchestra.run.smithi073.stdout:3311 write 0x149250a thru 0x149eb8e (0xc685 bytes) 2024-09-21T14:22:02.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:3312 writesame 0xd330fc3 thru 0xd33b011 data_size 0x1(0xa04f bytes) 2024-09-21T14:22:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:3313 read 0x294ee1e thru 0x294fb22 (0xd05 bytes) 2024-09-21T14:22:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:3314 write 0x1f23cc1 thru 0x1f2c976 (0x8cb6 bytes) 2024-09-21T14:22:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:3316 write 0xbd27ca5 thru 0xbd32487 (0xa7e3 bytes) 2024-09-21T14:22:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:3317 write 0x199de8e thru 0x19a498a (0x6afd bytes) 2024-09-21T14:22:02.148 INFO:teuthology.orchestra.run.smithi073.stdout:3318 punch from 0x3d0ac8 to 0x3dd91c, (0xce54 bytes) 2024-09-21T14:22:02.148 INFO:teuthology.orchestra.run.smithi073.stdout:3320 trunc from 0xd33b012 to 0x8082c44 2024-09-21T14:22:02.148 INFO:teuthology.orchestra.run.smithi073.stdout:3323 writesame 0x8ffd512 thru 0x8ffd589 data_size 0x6(0x78 bytes) 2024-09-21T14:22:02.148 INFO:teuthology.orchestra.run.smithi073.stdout:3324 write 0x63b180b thru 0x63b7282 (0x5a78 bytes) 2024-09-21T14:22:02.148 INFO:teuthology.orchestra.run.smithi073.stdout:3325 trunc from 0x8ffd58a to 0x3f5c9b 2024-09-21T14:22:02.148 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:02.146+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:02.151 INFO:teuthology.orchestra.run.smithi073.stdout:3327 write 0xbc6bb78 thru 0xbc6d360 (0x17e9 bytes) 2024-09-21T14:22:02.161 INFO:teuthology.orchestra.run.smithi073.stdout:3328 trunc from 0xbc6d361 to 0x55bc4cc 2024-09-21T14:22:02.170 INFO:teuthology.orchestra.run.smithi073.stdout:3330 write 0xdc1224 thru 0xdcf453 (0xe230 bytes) 2024-09-21T14:22:02.172 INFO:teuthology.orchestra.run.smithi073.stdout:3331 writesame 0x4adce11 thru 0x4ae4158 data_size 0x7c(0x7348 bytes) 2024-09-21T14:22:02.177 INFO:teuthology.orchestra.run.smithi073.stdout:3332 trunc from 0x55bc4cc to 0x3d2ce91 2024-09-21T14:22:02.182 INFO:teuthology.orchestra.run.smithi073.stdout:3336 writesame 0x314e45c thru 0x315d27d data_size 0x16(0xee22 bytes) 2024-09-21T14:22:02.184 INFO:teuthology.orchestra.run.smithi073.stdout:3338 punch from 0x20db1bd to 0x20dd89a, (0x26dd bytes) 2024-09-21T14:22:02.184 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.184 INFO:teuthology.orchestra.run.smithi073.stdout:3340 punch from 0x5cd105 to 0x5d8aae, (0xb9a9 bytes) 2024-09-21T14:22:02.199 INFO:teuthology.orchestra.run.smithi073.stdout:3342 write 0xc07feaf thru 0xc0877e9 (0x793b bytes) 2024-09-21T14:22:02.202 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.202 INFO:teuthology.orchestra.run.smithi073.stdout:3344 clone 29 order 24 su 2097152 sc 14 2024-09-21T14:22:02.247 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:02.247+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:02.260 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:02.260 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:02.260 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:02.260 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:02.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:02.353 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:02.353+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:02.397 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:22:02.416 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:02.416 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:02.416 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:02.416 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:02.416 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:02.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:02.449 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:02.450+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:02.551 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:02.551+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.183285841999997, 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.183274941000001, 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:02.584 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.183264532999999, 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:02.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:02.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:02.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:02.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.183256591999999, 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:02.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106518606999998, 2024-09-21T14:22:02.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:02.599 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106500197000003, 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106474448, 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:02.603 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:02.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106453418999997, 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.608 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:02.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:02.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:02.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106427568999997, 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.612 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:02.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.10640909, 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.616 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:02.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:02.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106390967999999, 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.620 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:02.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:02.624 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106370605999999, 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106350913999997, 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.632 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106333935999999, 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:02.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106314263999998, 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:02.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:02.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:02.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106284801000001, 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:02.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106242555000001, 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:02.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:02.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:02.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106223567000001, 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.650 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106200209000001, 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:02.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:02.658 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.106176855999998, 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:02.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:02.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:02.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:02.663 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:02.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:02.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:02.665 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T14:22:02.665 INFO:teuthology.orchestra.run.smithi073.stdout:3345 read 0xa02542a thru 0xa02ec5d (0x9834 bytes) 2024-09-21T14:22:02.665 INFO:teuthology.orchestra.run.smithi073.stdout:3347 trunc from 0xc0877ea to 0x73eb93e 2024-09-21T14:22:02.666 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:02.664+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:3348 read 0x10746d3 thru 0x107f23f (0xab6d bytes) 2024-09-21T14:22:02.669 INFO:teuthology.orchestra.run.smithi073.stdout:3349 read 0x126a0c0 thru 0x12701f6 (0x6137 bytes) 2024-09-21T14:22:02.669 INFO:teuthology.orchestra.run.smithi073.stdout:3350 read 0x6627da0 thru 0x663729d (0xf4fe bytes) 2024-09-21T14:22:02.680 INFO:teuthology.orchestra.run.smithi073.stdout:3354 write 0xc860304 thru 0xc8655f4 (0x52f1 bytes) 2024-09-21T14:22:02.682 INFO:teuthology.orchestra.run.smithi073.stdout:3355 punch from 0x44457db to 0x444ca7e, (0x72a3 bytes) 2024-09-21T14:22:02.686 INFO:teuthology.orchestra.run.smithi073.stdout:3356 trunc from 0xc8655f5 to 0x934a5de 2024-09-21T14:22:02.694 INFO:teuthology.orchestra.run.smithi073.stdout:3358 read 0x1e7559b thru 0x1e79428 (0x3e8e bytes) 2024-09-21T14:22:02.695 INFO:teuthology.orchestra.run.smithi073.stdout:3359 read 0x364a919 thru 0x3658359 (0xda41 bytes) 2024-09-21T14:22:02.695 INFO:teuthology.orchestra.run.smithi073.stdout:3360 read 0x72a52b thru 0x72c854 (0x232a bytes) 2024-09-21T14:22:02.696 INFO:teuthology.orchestra.run.smithi073.stdout:3362 punch from 0x5227a16 to 0x52339d2, (0xbfbc bytes) 2024-09-21T14:22:02.696 INFO:teuthology.orchestra.run.smithi073.stdout:3364 write 0x7d8311f thru 0x7d8c06d (0x8f4f bytes) 2024-09-21T14:22:02.710 INFO:teuthology.orchestra.run.smithi073.stdout:3365 writesame 0xe8945a3 thru 0xe897306 data_size 0x23(0x2d64 bytes) 2024-09-21T14:22:02.710 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.710 INFO:teuthology.orchestra.run.smithi073.stdout:3367 punch from 0x78daa9f to 0x78de497, (0x39f8 bytes) 2024-09-21T14:22:02.710 INFO:teuthology.orchestra.run.smithi073.stdout:3368 read 0xacbaf39 thru 0xacc8d28 (0xddf0 bytes) 2024-09-21T14:22:02.711 INFO:teuthology.orchestra.run.smithi073.stdout:3369 write 0x8099b68 thru 0x80a626d (0xc706 bytes) 2024-09-21T14:22:02.723 INFO:teuthology.orchestra.run.smithi073.stdout:3370 trunc from 0xe897307 to 0x54a6b54 2024-09-21T14:22:02.730 INFO:teuthology.orchestra.run.smithi073.stdout:3374 writesame 0x336991c thru 0x33717f0 data_size 0x3(0x7ed5 bytes) 2024-09-21T14:22:02.737 INFO:teuthology.orchestra.run.smithi073.stdout:3375 trunc from 0x54a6b54 to 0x4bceef1 2024-09-21T14:22:02.743 INFO:teuthology.orchestra.run.smithi073.stdout:3376 read 0x38c60a5 thru 0x38cbc41 (0x5b9d bytes) 2024-09-21T14:22:02.744 INFO:teuthology.orchestra.run.smithi073.stdout:3377 read 0x3da90e9 thru 0x3dacc35 (0x3b4d bytes) 2024-09-21T14:22:02.744 INFO:teuthology.orchestra.run.smithi073.stdout:3378 read 0x3336c8a thru 0x3342958 (0xbccf bytes) 2024-09-21T14:22:02.762 INFO:teuthology.orchestra.run.smithi073.stdout:3380 write 0xe60ac8c thru 0xe60ad03 (0x78 bytes) 2024-09-21T14:22:02.764 INFO:teuthology.orchestra.run.smithi073.stdout:3382 write 0x3866ecc thru 0x387315d (0xc292 bytes) 2024-09-21T14:22:02.765 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:02.765+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:02.766 INFO:teuthology.orchestra.run.smithi073.stdout:3384 write 0xa9004a9 thru 0xa906121 (0x5c79 bytes) 2024-09-21T14:22:02.774 INFO:teuthology.orchestra.run.smithi073.stdout:3385 write 0xed013c2 thru 0xed0f449 (0xe088 bytes) 2024-09-21T14:22:02.778 INFO:teuthology.orchestra.run.smithi073.stdout:3386 writesame 0x3550ad6 thru 0x35529db data_size 0x13(0x1f06 bytes) 2024-09-21T14:22:02.779 INFO:teuthology.orchestra.run.smithi073.stdout:3387 writesame 0xd0aa8bd thru 0xd0abffa data_size 0x11(0x173e bytes) 2024-09-21T14:22:02.781 INFO:teuthology.orchestra.run.smithi073.stdout:3388 write 0xd8ba8a6 thru 0xd8c3603 (0x8d5e bytes) 2024-09-21T14:22:02.783 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.783 INFO:teuthology.orchestra.run.smithi073.stdout:3390 punch from 0x770983f to 0x770b665, (0x1e26 bytes) 2024-09-21T14:22:02.783 INFO:teuthology.orchestra.run.smithi073.stdout:3391 punch from 0x42b4d4c to 0x42bfe5c, (0xb110 bytes) 2024-09-21T14:22:02.783 INFO:teuthology.orchestra.run.smithi073.stdout:3392 write 0xb51d696 thru 0xb52173d (0x40a8 bytes) 2024-09-21T14:22:02.784 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:02.789 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:02.789 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:02.789 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:02.789 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:02.794 INFO:teuthology.orchestra.run.smithi073.stdout:3394 trunc from 0xed0f44a to 0x656533e 2024-09-21T14:22:02.803 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:02.808 INFO:teuthology.orchestra.run.smithi073.stdout:3395 read 0x1db911b thru 0x1dbaecd (0x1db3 bytes) 2024-09-21T14:22:02.808 INFO:teuthology.orchestra.run.smithi073.stdout:3396 write 0x49a5dee thru 0x49af568 (0x977b bytes) 2024-09-21T14:22:02.810 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:02.810 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:02.815 INFO:teuthology.orchestra.run.smithi073.stdout:3397 trunc from 0x656533e to 0xb0d26b8 2024-09-21T14:22:02.820 INFO:teuthology.orchestra.run.smithi073.stdout:3398 read 0x9cf34c0 thru 0x9cf9409 (0x5f4a bytes) 2024-09-21T14:22:02.830 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:02.830 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:02.832 INFO:teuthology.orchestra.run.smithi073.stdout:3399 trunc from 0xb0d26b8 to 0x1880bb0 2024-09-21T14:22:02.849 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:02.850 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:02.863 INFO:teuthology.orchestra.run.smithi073.stdout:3400 write 0xe8853ac thru 0xe88991b (0x4570 bytes) 2024-09-21T14:22:02.866 INFO:teuthology.orchestra.run.smithi073.stdout:3401 write 0x279fade thru 0x27a2961 (0x2e84 bytes) 2024-09-21T14:22:02.866 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:02.866+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:02.867 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:02.867 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:02.867 INFO:teuthology.orchestra.run.smithi073.stdout:3403 write 0x97334b6 thru 0x973d94f (0xa49a bytes) 2024-09-21T14:22:02.868 INFO:teuthology.orchestra.run.smithi073.stdout:3407 writesame 0x7578a92 thru 0x758462a data_size 0x55(0xbb99 bytes) 2024-09-21T14:22:02.870 INFO:teuthology.orchestra.run.smithi073.stdout:truncating to largest ever: 0xee438fb 2024-09-21T14:22:02.870 INFO:teuthology.orchestra.run.smithi073.stdout:3408 trunc from 0xe88991c to 0xee438fb 2024-09-21T14:22:02.884 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:02.884 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:02.892 INFO:teuthology.orchestra.run.smithi073.stdout:3409 trunc from 0xee438fb to 0xc0a07d 2024-09-21T14:22:02.902 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:02.919 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:02.919 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:02.930 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:02.930 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:02.933 INFO:teuthology.orchestra.run.smithi073.stdout:3411 trunc from 0xc0a07d to 0xc2b2eac 2024-09-21T14:22:02.936 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:02.936 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:02.939 INFO:teuthology.orchestra.run.smithi073.stdout:3412 clone 30 order 21 su 131072 sc 2 2024-09-21T14:22:02.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:02.967 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:02.967+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:03.068 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:03.069+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:03.090 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.090 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.099 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:03.168 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:03.169+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:03.203 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.203 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:03.270 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:03.270+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:03.356 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.356 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.364 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:03.370 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:03.370+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:03.471 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:03.472+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:03.472 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone29 from 0xc2b2eac (overlap 0xc0a07d) to 0x9a19fd 2024-09-21T14:22:03.474 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.474 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:03.572 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:03.572+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:03.591 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T14:22:03.628 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.628 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:03.673 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:03.673+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:03.739 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.739 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:03.773 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:03.773+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:03.799 INFO:teuthology.orchestra.run.smithi073.stdout:3413 read 0x6f44b3f thru 0x6f4baf6 (0x6fb8 bytes) 2024-09-21T14:22:03.800 INFO:teuthology.orchestra.run.smithi073.stdout:3415 read 0x5e264d7 thru 0x5e2f2ba (0x8de4 bytes) 2024-09-21T14:22:03.800 INFO:teuthology.orchestra.run.smithi073.stdout:3418 read 0x6f8704d thru 0x6f8b819 (0x47cd bytes) 2024-09-21T14:22:03.801 INFO:teuthology.orchestra.run.smithi073.stdout:3419 write 0x22e1948 thru 0x22f0fdb (0xf694 bytes) 2024-09-21T14:22:03.810 INFO:teuthology.orchestra.run.smithi073.stdout:3420 writesame 0x4dc6d19 thru 0x4dce561 data_size 0x7(0x7849 bytes) 2024-09-21T14:22:03.814 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:03.814 INFO:teuthology.orchestra.run.smithi073.stdout:3422 read 0xa67d906 thru 0xa68a458 (0xcb53 bytes) 2024-09-21T14:22:03.814 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:03.814 INFO:teuthology.orchestra.run.smithi073.stdout:3424 flatten 2024-09-21T14:22:03.859 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.859 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.860 INFO:teuthology.orchestra.run.smithi073.stdout:3425 write 0x371fda6 thru 0x3723297 (0x34f2 bytes) 2024-09-21T14:22:03.866 INFO:teuthology.orchestra.run.smithi073.stdout:3426 read 0x39a51ad thru 0x39a9cc0 (0x4b14 bytes) 2024-09-21T14:22:03.866 INFO:teuthology.orchestra.run.smithi073.stdout:3427 writesame 0x542f335 thru 0x54346cd data_size 0x1(0x5399 bytes) 2024-09-21T14:22:03.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:03.874 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:03.874+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:03.877 INFO:teuthology.orchestra.run.smithi073.stdout:3428 writesame 0x279e7ec thru 0x27a5e24 data_size 0x1(0x7639 bytes) 2024-09-21T14:22:03.894 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:03.894 INFO:teuthology.orchestra.run.smithi073.stdout:3433 read 0xad13d21 thru 0xad239fd (0xfcdd bytes) 2024-09-21T14:22:03.894 INFO:teuthology.orchestra.run.smithi073.stdout:3435 writesame 0x34a88d4 thru 0x34b087b data_size 0x14(0x7fa8 bytes) 2024-09-21T14:22:03.901 INFO:teuthology.orchestra.run.smithi073.stdout:3437 trunc from 0xc2b2eac to 0x757e4d2 2024-09-21T14:22:03.907 INFO:teuthology.orchestra.run.smithi073.stdout:3438 read 0x37464b8 thru 0x3749110 (0x2c59 bytes) 2024-09-21T14:22:03.910 INFO:teuthology.orchestra.run.smithi073.stdout:3440 trunc from 0x757e4d2 to 0x491f8d6 2024-09-21T14:22:03.922 INFO:teuthology.orchestra.run.smithi073.stdout:3442 trunc from 0x491f8d6 to 0x2efa397 2024-09-21T14:22:03.951 INFO:teuthology.orchestra.run.smithi073.stdout:3446 write 0xea58cb0 thru 0xea657cd (0xcb1e bytes) 2024-09-21T14:22:03.954 INFO:teuthology.orchestra.run.smithi073.stdout:3447 punch from 0x2ab6cb4 to 0x2abe132, (0x747e bytes) 2024-09-21T14:22:03.969 INFO:teuthology.orchestra.run.smithi073.stdout:3448 trunc from 0xea657ce to 0x2784652 2024-09-21T14:22:03.975 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:03.975+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:03.979 INFO:teuthology.orchestra.run.smithi073.stdout:3451 read 0x251f16f thru 0x2523ea4 (0x4d36 bytes) 2024-09-21T14:22:03.980 INFO:teuthology.orchestra.run.smithi073.stdout:3454 punch from 0xb6c36e to 0xb6fcb0, (0x3942 bytes) 2024-09-21T14:22:03.980 INFO:teuthology.orchestra.run.smithi073.stdout:3455 punch from 0x98bc92 to 0x98f257, (0x35c5 bytes) 2024-09-21T14:22:03.980 INFO:teuthology.orchestra.run.smithi073.stdout:3456 punch from 0x8fd156 to 0x9060ee, (0x8f98 bytes) 2024-09-21T14:22:03.981 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:03.981 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:03.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:03.991 INFO:teuthology.orchestra.run.smithi073.stdout:3460 trunc from 0x2784652 to 0xc352582 2024-09-21T14:22:04.009 INFO:teuthology.orchestra.run.smithi073.stdout:3461 trunc from 0xc352582 to 0x179d51d 2024-09-21T14:22:04.036 INFO:teuthology.orchestra.run.smithi073.stdout:3462 writesame 0xc514d0b thru 0xc51f1c9 data_size 0x19(0xa4bf bytes) 2024-09-21T14:22:04.036 INFO:teuthology.orchestra.run.smithi073.stdout:3464 write 0x4380b24 thru 0x438da0e (0xceeb bytes) 2024-09-21T14:22:04.039 INFO:teuthology.orchestra.run.smithi073.stdout:3466 write 0x57c336a thru 0x57c5afc (0x2793 bytes) 2024-09-21T14:22:04.042 INFO:teuthology.orchestra.run.smithi073.stdout:3467 write 0xa0c2e99 thru 0xa0cba81 (0x8be9 bytes) 2024-09-21T14:22:04.048 INFO:teuthology.orchestra.run.smithi073.stdout:3471 trunc from 0xc51f1ca to 0x9afe320 2024-09-21T14:22:04.058 INFO:teuthology.orchestra.run.smithi073.stdout:3472 write 0x4e44604 thru 0x4e52617 (0xe014 bytes) 2024-09-21T14:22:04.061 INFO:teuthology.orchestra.run.smithi073.stdout:3473 writesame 0xaa7507 thru 0xaabb49 data_size 0x1(0x4643 bytes) 2024-09-21T14:22:04.071 INFO:teuthology.orchestra.run.smithi073.stdout:3474 read 0x1b9274b thru 0x1b98475 (0x5d2b bytes) 2024-09-21T14:22:04.071 INFO:teuthology.orchestra.run.smithi073.stdout:3476 read 0x2f28102 thru 0x2f2e13a (0x6039 bytes) 2024-09-21T14:22:04.071 INFO:teuthology.orchestra.run.smithi073.stdout:3478 read 0x39bda56 thru 0x39ccc31 (0xf1dc bytes) 2024-09-21T14:22:04.071 INFO:teuthology.orchestra.run.smithi073.stdout:3479 write 0x6a56cc5 thru 0x6a5ed7c (0x80b8 bytes) 2024-09-21T14:22:04.076 INFO:teuthology.orchestra.run.smithi073.stdout:3481 trunc from 0x9afe320 to 0x80dbfb4 2024-09-21T14:22:04.076 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:04.076+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:04.081 INFO:teuthology.orchestra.run.smithi073.stdout:3482 punch from 0x3fa4e07 to 0x3fa5ff3, (0x11ec bytes) 2024-09-21T14:22:04.081 INFO:teuthology.orchestra.run.smithi073.stdout:3483 write 0x9a55e9 thru 0x9a94f7 (0x3f0f bytes) 2024-09-21T14:22:04.083 INFO:teuthology.orchestra.run.smithi073.stdout:3484 read 0x2e110c1 thru 0x2e1ad82 (0x9cc2 bytes) 2024-09-21T14:22:04.083 INFO:teuthology.orchestra.run.smithi073.stdout:3485 punch from 0x4de6d9b to 0x4de76d3, (0x938 bytes) 2024-09-21T14:22:04.083 INFO:teuthology.orchestra.run.smithi073.stdout:3486 read 0x2e220d1 thru 0x2e2bf43 (0x9e73 bytes) 2024-09-21T14:22:04.083 INFO:teuthology.orchestra.run.smithi073.stdout:3487 punch from 0x32bdd89 to 0x32cb9a5, (0xdc1c bytes) 2024-09-21T14:22:04.083 INFO:teuthology.orchestra.run.smithi073.stdout:3488 read 0x2635963 thru 0x263943c (0x3ada bytes) 2024-09-21T14:22:04.093 INFO:teuthology.orchestra.run.smithi073.stdout:3489 writesame 0xb7422f1 thru 0xb74cdbc data_size 0x22(0xaacc bytes) 2024-09-21T14:22:04.093 INFO:teuthology.orchestra.run.smithi073.stdout:3491 punch from 0x51a2b20 to 0x51a7dc2, (0x52a2 bytes) 2024-09-21T14:22:04.093 INFO:teuthology.orchestra.run.smithi073.stdout:3492 writesame 0x8f96dec thru 0x8fa281d data_size 0x1(0xba32 bytes) 2024-09-21T14:22:04.101 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.101 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.113 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:04.116 INFO:teuthology.orchestra.run.smithi073.stdout:3493 punch from 0xa1d3cb2 to 0xa1e2abc, (0xee0a bytes) 2024-09-21T14:22:04.116 INFO:teuthology.orchestra.run.smithi073.stdout:3495 write 0x11030e4 thru 0x1111841 (0xe75e bytes) 2024-09-21T14:22:04.119 INFO:teuthology.orchestra.run.smithi073.stdout:3496 write 0x7a9ca6d thru 0x7a9febf (0x3453 bytes) 2024-09-21T14:22:04.122 INFO:teuthology.orchestra.run.smithi073.stdout:3497 read 0x9f13ac1 thru 0x9f16bbb (0x30fb bytes) 2024-09-21T14:22:04.122 INFO:teuthology.orchestra.run.smithi073.stdout:3498 write 0x4fcbefa thru 0x4fd9133 (0xd23a bytes) 2024-09-21T14:22:04.135 INFO:teuthology.orchestra.run.smithi073.stdout:3499 trunc from 0xb74cdbd to 0x19dade0 2024-09-21T14:22:04.148 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.157 INFO:teuthology.orchestra.run.smithi073.stdout:3502 writesame 0x458aa13 thru 0x45918b9 data_size 0x1(0x6ea7 bytes) 2024-09-21T14:22:04.157 INFO:teuthology.orchestra.run.smithi073.stdout:3503 read 0x1ae50fd thru 0x1af3fda (0xeede bytes) 2024-09-21T14:22:04.165 INFO:teuthology.orchestra.run.smithi073.stdout:3506 write 0x630d536 thru 0x630e2a9 (0xd74 bytes) 2024-09-21T14:22:04.170 INFO:teuthology.orchestra.run.smithi073.stdout:3507 read 0x3e2a6b3 thru 0x3e3545e (0xadac bytes) 2024-09-21T14:22:04.177 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:04.177+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:04.181 INFO:teuthology.orchestra.run.smithi073.stdout:3508 write 0xb8783bb thru 0xb88068e (0x82d4 bytes) 2024-09-21T14:22:04.189 INFO:teuthology.orchestra.run.smithi073.stdout:3510 writesame 0x9d4329f thru 0x9d4a092 data_size 0x3e(0x6df4 bytes) 2024-09-21T14:22:04.193 INFO:teuthology.orchestra.run.smithi073.stdout:3511 writesame 0x9ad4700 thru 0x9ae3c2e data_size 0x1(0xf52f bytes) 2024-09-21T14:22:04.216 INFO:teuthology.orchestra.run.smithi073.stdout:3512 read 0x50f1e74 thru 0x50fbe33 (0x9fc0 bytes) 2024-09-21T14:22:04.217 INFO:teuthology.orchestra.run.smithi073.stdout:3513 writesame 0x1513ba0 thru 0x151d6d8 data_size 0x1(0x9b39 bytes) 2024-09-21T14:22:04.220 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.220 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.229 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:04.232 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.232 INFO:teuthology.orchestra.run.smithi073.stdout:3516 write 0x56e6cba thru 0x56e76ff (0xa46 bytes) 2024-09-21T14:22:04.236 INFO:teuthology.orchestra.run.smithi073.stdout:3517 writesame 0x7b97b4e thru 0x7b9d24f data_size 0x4a(0x5702 bytes) 2024-09-21T14:22:04.242 INFO:teuthology.orchestra.run.smithi073.stdout:3518 read 0x8162e99 thru 0x8163f11 (0x1079 bytes) 2024-09-21T14:22:04.242 INFO:teuthology.orchestra.run.smithi073.stdout:3519 clone 31 order 18 su 16384 sc 3 2024-09-21T14:22:04.278 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:04.278+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:04.331 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.331 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.339 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:04.379 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:04.379+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:04.412 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:22:04.445 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.445 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.454 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:04.498 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:04.498+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:04.560 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.560 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:04.610 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:04.610+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:04.631 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T14:22:04.645 INFO:teuthology.orchestra.run.smithi073.stdout:3520 punch from 0x37dd6bc to 0x37e5734, (0x8078 bytes) 2024-09-21T14:22:04.655 INFO:teuthology.orchestra.run.smithi073.stdout:3521 write 0xbefb2d3 thru 0xbf05fc2 (0xacf0 bytes) 2024-09-21T14:22:04.667 INFO:teuthology.orchestra.run.smithi073.stdout:3522 trunc from 0xbf05fc3 to 0x5dad296 2024-09-21T14:22:04.681 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:04.681+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:04.682 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.682 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.683 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.683 INFO:teuthology.orchestra.run.smithi073.stdout:3524 write 0x15f99a2 thru 0x15fac74 (0x12d3 bytes) 2024-09-21T14:22:04.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:04.701 INFO:teuthology.orchestra.run.smithi073.stdout:3525 write 0xc8c8bc9 thru 0xc8c9653 (0xa8b bytes) 2024-09-21T14:22:04.704 INFO:teuthology.orchestra.run.smithi073.stdout:3527 read 0xa9673d5 thru 0xa973e19 (0xca45 bytes) 2024-09-21T14:22:04.704 INFO:teuthology.orchestra.run.smithi073.stdout:3528 write 0xabb2202 thru 0xabb2c04 (0xa03 bytes) 2024-09-21T14:22:04.707 INFO:teuthology.orchestra.run.smithi073.stdout:3531 read 0x3406c60 thru 0x340e0e6 (0x7487 bytes) 2024-09-21T14:22:04.714 INFO:teuthology.orchestra.run.smithi073.stdout:3532 trunc from 0xc8c9654 to 0x71906d2 2024-09-21T14:22:04.734 INFO:teuthology.orchestra.run.smithi073.stdout:3534 writesame 0xad7031e thru 0xad73808 data_size 0x1(0x34eb bytes) 2024-09-21T14:22:04.734 INFO:teuthology.orchestra.run.smithi073.stdout:3535 read 0x1ee953c thru 0x1ef2762 (0x9227 bytes) 2024-09-21T14:22:04.734 INFO:teuthology.orchestra.run.smithi073.stdout:3538 punch from 0x5f8876e to 0x5f8bbe7, (0x3479 bytes) 2024-09-21T14:22:04.745 INFO:teuthology.orchestra.run.smithi073.stdout:3541 trunc from 0xad73809 to 0x171ffa5 2024-09-21T14:22:04.770 INFO:teuthology.orchestra.run.smithi073.stdout:3543 writesame 0x28d04a8 thru 0x28d479f data_size 0x4(0x42f8 bytes) 2024-09-21T14:22:04.770 INFO:teuthology.orchestra.run.smithi073.stdout:3544 writesame 0x261030 thru 0x26e7bd data_size 0x33(0xd78e bytes) 2024-09-21T14:22:04.782 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:04.782+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:04.817 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.817 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.820 INFO:teuthology.orchestra.run.smithi073.stdout:3545 read 0x175429b thru 0x175e2b2 (0xa018 bytes) 2024-09-21T14:22:04.820 INFO:teuthology.orchestra.run.smithi073.stdout:3546 read 0x1dc492b thru 0x1dce0e2 (0x97b8 bytes) 2024-09-21T14:22:04.820 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:04.839 INFO:teuthology.orchestra.run.smithi073.stdout:3548 write 0xdf4753f thru 0xdf4d4e5 (0x5fa7 bytes) 2024-09-21T14:22:04.845 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.845 INFO:teuthology.orchestra.run.smithi073.stdout:3552 punch from 0x6e782d3 to 0x6e79c07, (0x1934 bytes) 2024-09-21T14:22:04.845 INFO:teuthology.orchestra.run.smithi073.stdout:3553 write 0x7ffabe thru 0x809f5c (0xa49f bytes) 2024-09-21T14:22:04.861 INFO:teuthology.orchestra.run.smithi073.stdout:3554 trunc from 0xdf4d4e6 to 0x63d2818 2024-09-21T14:22:04.882 INFO:teuthology.orchestra.run.smithi073.stdout:3555 writesame 0xa269000 thru 0xa26ae22 data_size 0x1(0x1e23 bytes) 2024-09-21T14:22:04.882 INFO:teuthology.orchestra.run.smithi073.stdout:3558 read 0x7abc6df thru 0x7abd7c9 (0x10eb bytes) 2024-09-21T14:22:04.882 INFO:teuthology.orchestra.run.smithi073.stdout:3559 write 0x9b30e2a thru 0x9b34b86 (0x3d5d bytes) 2024-09-21T14:22:04.882 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:04.882+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:04.887 INFO:teuthology.orchestra.run.smithi073.stdout:3560 read 0x78e6be0 thru 0x78efbdc (0x8ffd bytes) 2024-09-21T14:22:04.887 INFO:teuthology.orchestra.run.smithi073.stdout:3564 writesame 0xb62e13 thru 0xb72c95 data_size 0x53(0xfe83 bytes) 2024-09-21T14:22:04.898 INFO:teuthology.orchestra.run.smithi073.stdout:3565 trunc from 0xa26ae23 to 0x8c36396 2024-09-21T14:22:04.907 INFO:teuthology.orchestra.run.smithi073.stdout:3566 write 0x183e4da thru 0x184e000 (0xfb27 bytes) 2024-09-21T14:22:04.912 INFO:teuthology.orchestra.run.smithi073.stdout:3567 write 0x61f81a2 thru 0x61fd6cf (0x552e bytes) 2024-09-21T14:22:04.916 INFO:teuthology.orchestra.run.smithi073.stdout:3570 punch from 0xf19373 to 0xf2423b, (0xaec8 bytes) 2024-09-21T14:22:04.916 INFO:teuthology.orchestra.run.smithi073.stdout:3571 read 0x60e232 thru 0x61e076 (0xfe45 bytes) 2024-09-21T14:22:04.916 INFO:teuthology.orchestra.run.smithi073.stdout:3572 write 0x7bdb6f0 thru 0x7be7750 (0xc061 bytes) 2024-09-21T14:22:04.921 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.921 INFO:teuthology.orchestra.run.smithi073.stdout:3574 writesame 0x5e0b987 thru 0x5e0cf5d data_size 0x1(0x15d7 bytes) 2024-09-21T14:22:04.925 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:04.925 INFO:teuthology.orchestra.run.smithi073.stdout:3576 read 0x5db8c5 thru 0x5e319a (0x78d6 bytes) 2024-09-21T14:22:04.925 INFO:teuthology.orchestra.run.smithi073.stdout:3577 clone 32 order 25 su 131072 sc 10 2024-09-21T14:22:04.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:04.949 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:04.949 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:04.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:04.983 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:04.984+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:05.069 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:05.069 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:05.084 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:05.084+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:05.086 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:05.086 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:05.102 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:05.102 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:05.104 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:05.104 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.113 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:05.118 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:05.118 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:05.134 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:05.134 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:05.151 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:05.167 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:05.167 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:05.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:05.184 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:05.184 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:05.184+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:05.221 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.221 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:05.222 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:05.222 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:05.222 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:05.285 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:05.286+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:05.374 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.374 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.153303637, 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:05.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:05.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:05.379 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42293 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3274752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.310888638000002, 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032075181000000001, 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42293 2024-09-21T14:22:05.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063351+0000", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063354+0000", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0880000000000002e-06 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063359+0000", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7620000000000001e-06 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063364+0000", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3560000000000003e-06 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095339+0000", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031974925000000001 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095389+0000", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0095000000000001e-05 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095427+0000", 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8389000000000001e-05 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42294 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3500032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.310873033, 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032160279, 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.384 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42294 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063369+0000", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0320000000000002e-06 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063375+0000", 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4729999999999999e-06 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063379+0000", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5329999999999996e-06 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095440+0000", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032060937999999997 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095489+0000", 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8711000000000002e-05 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095528+0000", 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8926e-05 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53665 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6615125800000001, 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032083390000000003, 2024-09-21T14:22:05.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53665 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712733+0000", 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712740+0000", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9290000000000001e-06 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712746+0000", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744674+0000", 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031927981000000001 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744749+0000", 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744811+0000", 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2681000000000005e-05 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6614791750000002, 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:05.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:05.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.661461471, 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:05.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:05.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6614458000000001, 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6614302859999999, 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.408 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6614152190000002, 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:05.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6614000239999998, 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.413 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:05.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:05.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6613849179999995, 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.417 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:05.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:05.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6613698079999999, 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:05.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6613544430000005, 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:05.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.429 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6613389840000004, 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.433 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.661323951, 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:05.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:05.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.437 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6613080619999998, 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.441 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6612823859999999, 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:05.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:05.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.445 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.661267101, 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:05.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:05.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:05.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.449 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6612451369999999, 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:05.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:05.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.6612291539999999, 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.458 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.458 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.458 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:05.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:05.458 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:22:05.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:22:05.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:22:05.461 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:05.459+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:05.483 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone31 from 0x8c36396 (overlap 0x171ffa5) to 0x0 2024-09-21T14:22:05.560 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:05.560+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:05.622 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T14:22:05.623 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.623 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:05.623 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:05.623 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:05.678 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:05.678+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:05.762 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:05.762+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:05.786 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.786 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:05.786 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:05.786 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:05.786 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.795 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 63, 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:22:05.421464+0000", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:21:48.316234+0000", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5801984, 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:22:05.806 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:22:05.807 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:05.808 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:05.809 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:22:05.810 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:05.811 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:05.812 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.9900000095367432, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.9900000095367432, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.75, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.75, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:05.813 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:05.814 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:05.815 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:05.816 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:05.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:22:05.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.821 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "63", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 35, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 63, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:05.822 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:05.823 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:05.824 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.7400000095367432, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.7400000095367432, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:22:05.825 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:22:05.826 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.829 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 33, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.831 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.832 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.833 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.834 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:22:05.835 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.836 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:22:05.840 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:22:05.841 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.843 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:22:05.844 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.845 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.846 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.847 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.849 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.850 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.851 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.852 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 47, 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.853 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.854 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.855 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.856 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:05.857 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.858 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.860 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.861 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:22:05.862 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:05.863 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.864 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.865 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:05.867 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:22:05.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.869 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.870 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.871 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.d", 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: 2 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:22:05.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.876 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.878 INFO:tasks.thrashosds.thrasher:j is {'epoch': 63, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:22:05.421464+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:21:48.316234+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.d', 'osds': [6, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:22:05.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3578 writesame 0x7038cc5 thru 0x703c099 data_size 0x1(0x33d5 bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3580 punch from 0x8c277af to 0x8c33add, (0xc32e bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3582 writesame 0x8b1bb28 thru 0x8b1e5e1 data_size 0x3(0x2aba bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3585 write 0xe0201d thru 0xe04404 (0x23e8 bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3587 read 0x162c04f thru 0x1635524 (0x94d6 bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3588 read 0x7c6e2a4 thru 0x7c70b90 (0x28ed bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3589 punch from 0x6cdaad9 to 0x6cdb413, (0x93a bytes) 2024-09-21T14:22:05.880 INFO:teuthology.orchestra.run.smithi073.stdout:3590 write 0xcd684b4 thru 0xcd73abb (0xb608 bytes) 2024-09-21T14:22:05.881 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:05.878+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:05.886 INFO:teuthology.orchestra.run.smithi073.stdout:3591 write 0x1c5dff6 thru 0x1c6a496 (0xc4a1 bytes) 2024-09-21T14:22:05.899 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:05.899 INFO:teuthology.orchestra.run.smithi073.stdout:3593 punch from 0xbd3749b to 0xbd3e712, (0x7277 bytes) 2024-09-21T14:22:05.899 INFO:teuthology.orchestra.run.smithi073.stdout:3596 write 0x94c3194 thru 0x94cbe41 (0x8cae bytes) 2024-09-21T14:22:05.912 INFO:teuthology.orchestra.run.smithi073.stdout:3597 write 0xca5f674 thru 0xca60168 (0xaf5 bytes) 2024-09-21T14:22:05.913 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:05.913 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:05.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:05.913 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:05.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.520160017999999, 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:05.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.917 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.520149117000003, 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.918 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:05.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.921 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.520138709000001, 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.922 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:05.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:05.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.520130768000001, 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.926 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443392783, 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.930 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:05.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:05.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443374372999997, 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:05.934 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.938 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443348624000002, 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.942 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443327595, 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443301744999999, 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.947 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.950 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443283266000002, 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.954 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443265144000001, 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.955 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:05.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443244782000001, 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:05.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443225089999999, 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.963 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:05.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443208112000001, 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:05.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:05.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:05.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.44318844, 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:05.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443158977000003, 2024-09-21T14:22:05.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443116731000003, 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:05.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443097743000003, 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:05.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:05.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:05.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:05.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443074385000003, 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:05.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.443051032, 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:05.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:05.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:05.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:05.996 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:05.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:05.999 INFO:teuthology.orchestra.run.smithi073.stdout:3598 write 0x8353507 thru 0x835c409 (0x8f03 bytes) 2024-09-21T14:22:05.999 INFO:teuthology.orchestra.run.smithi073.stdout:3600 write 0x93a6ebe thru 0x93adf16 (0x7059 bytes) 2024-09-21T14:22:05.999 INFO:teuthology.orchestra.run.smithi073.stdout:3601 write 0x219c79d thru 0x21a4815 (0x8079 bytes) 2024-09-21T14:22:05.999 INFO:teuthology.orchestra.run.smithi073.stdout:3603 write 0x145489 thru 0x154518 (0xf090 bytes) 2024-09-21T14:22:05.999 INFO:teuthology.orchestra.run.smithi073.stdout:3604 read 0x6571b9a thru 0x657e8c2 (0xcd29 bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3606 trunc from 0xcd73abc to 0x9c9c7bf 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3607 read 0x711c6a7 thru 0x7125e5f (0x97b9 bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3608 writesame 0x7ecd8fd thru 0x7ed4969 data_size 0x1(0x706d bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3611 read 0x44a2b5d thru 0x44a9f78 (0x741c bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3613 writesame 0x23cd319 thru 0x23ce89b data_size 0x1(0x1583 bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3614 read 0x212f749 thru 0x21366aa (0x6f62 bytes) 2024-09-21T14:22:06.000 INFO:teuthology.orchestra.run.smithi073.stdout:3616 punch from 0x740a4e8 to 0x7412982, (0x849a bytes) 2024-09-21T14:22:06.000 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:05.998+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:06.001 INFO:teuthology.orchestra.run.smithi073.stdout:3618 write 0xb29096e thru 0xb296bdc (0x626f bytes) 2024-09-21T14:22:06.003 INFO:teuthology.orchestra.run.smithi073.stdout:3619 read 0x714bc36 thru 0x714f6dd (0x3aa8 bytes) 2024-09-21T14:22:06.003 INFO:teuthology.orchestra.run.smithi073.stdout:3620 write 0x153c480 thru 0x154a241 (0xddc2 bytes) 2024-09-21T14:22:06.004 INFO:teuthology.orchestra.run.smithi073.stdout:3622 read 0x502e964 thru 0x5035f46 (0x75e3 bytes) 2024-09-21T14:22:06.004 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.004 INFO:teuthology.orchestra.run.smithi073.stdout:3624 writesame 0x8b8aac0 thru 0x8b98ea0 data_size 0x1(0xe3e1 bytes) 2024-09-21T14:22:06.037 INFO:teuthology.orchestra.run.smithi073.stdout:3625 write 0xcae7c94 thru 0xcaee5e9 (0x6956 bytes) 2024-09-21T14:22:06.038 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.038 INFO:teuthology.orchestra.run.smithi073.stdout:3627 read 0xc224bbe thru 0xc232778 (0xdbbb bytes) 2024-09-21T14:22:06.039 INFO:teuthology.orchestra.run.smithi073.stdout:3628 punch from 0x3433a46 to 0x343fecc, (0xc486 bytes) 2024-09-21T14:22:06.039 INFO:teuthology.orchestra.run.smithi073.stdout:3629 read 0xae04f47 thru 0xae0edf2 (0x9eac bytes) 2024-09-21T14:22:06.039 INFO:teuthology.orchestra.run.smithi073.stdout:3630 read 0x28cae21 thru 0x28cb669 (0x849 bytes) 2024-09-21T14:22:06.040 INFO:teuthology.orchestra.run.smithi073.stdout:3631 punch from 0x969cbe9 to 0x96ac4e0, (0xf8f7 bytes) 2024-09-21T14:22:06.047 INFO:teuthology.orchestra.run.smithi073.stdout:3632 trunc from 0xcaee5ea to 0x645b0f2 2024-09-21T14:22:06.058 INFO:teuthology.orchestra.run.smithi073.stdout:3633 punch from 0x10fc66d to 0x10ffb2e, (0x34c1 bytes) 2024-09-21T14:22:06.063 INFO:teuthology.orchestra.run.smithi073.stdout:3634 trunc from 0x645b0f2 to 0x216d7d9 2024-09-21T14:22:06.079 INFO:teuthology.orchestra.run.smithi073.stdout:3635 writesame 0x3eb9885 thru 0x3ec47e1 data_size 0x1(0xaf5d bytes) 2024-09-21T14:22:06.081 INFO:teuthology.orchestra.run.smithi073.stdout:3636 read 0x727635 thru 0x727a8a (0x456 bytes) 2024-09-21T14:22:06.082 INFO:teuthology.orchestra.run.smithi073.stdout:3637 punch from 0x1564b73 to 0x156d137, (0x85c4 bytes) 2024-09-21T14:22:06.082 INFO:teuthology.orchestra.run.smithi073.stdout:3638 read 0x1a425a3 thru 0x1a50610 (0xe06e bytes) 2024-09-21T14:22:06.088 INFO:teuthology.orchestra.run.smithi073.stdout:3640 write 0x55b6db6 thru 0x55c20d1 (0xb31c bytes) 2024-09-21T14:22:06.097 INFO:teuthology.orchestra.run.smithi073.stdout:3641 write 0x761f3e0 thru 0x762657b (0x719c bytes) 2024-09-21T14:22:06.098 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:06.098+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:06.099 INFO:teuthology.orchestra.run.smithi073.stdout:3642 read 0xe67be0 thru 0xe703ac (0x87cd bytes) 2024-09-21T14:22:06.099 INFO:teuthology.orchestra.run.smithi073.stdout:3643 read 0x30b95 thru 0x409e9 (0xfe55 bytes) 2024-09-21T14:22:06.100 INFO:teuthology.orchestra.run.smithi073.stdout:3644 punch from 0x205e70f to 0x2064748, (0x6039 bytes) 2024-09-21T14:22:06.104 INFO:teuthology.orchestra.run.smithi073.stdout:3645 trunc from 0x762657c to 0xa79d302 2024-09-21T14:22:06.109 INFO:teuthology.orchestra.run.smithi073.stdout:3646 write 0x48fab3 thru 0x497f5a (0x84a8 bytes) 2024-09-21T14:22:06.118 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:06.118 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:06.119 INFO:teuthology.orchestra.run.smithi073.stdout:3647 write 0xd36a979 thru 0xd37248a (0x7b12 bytes) 2024-09-21T14:22:06.120 INFO:teuthology.orchestra.run.smithi073.stdout:3648 punch from 0xbf080cb to 0xbf0b1b8, (0x30ed bytes) 2024-09-21T14:22:06.120 INFO:teuthology.orchestra.run.smithi073.stdout:3649 clone 33 order 23 su 131072 sc 7 2024-09-21T14:22:06.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:22:06.199 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:06.200+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:06.208 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-21T14:22:05.426196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1060431606,"num_objects":1278,"num_object_clones":321,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1278,"num_whiteouts":528,"num_read":87360,"num_read_kb":420088,"num_write":11161,"num_write_kb":180940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8977,"ondisk_log_size":8977,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":306084,"kb_used_data":33360,"kb_used_omap":10,"kb_used_meta":272629,"kb_avail":660296796,"statfs":{"total":676457349120,"available":676143919104,"internally_reserved":0,"allocated":34160640,"data_stored":256489577,"data_compressed":14278308,"data_compressed_allocated":16527360,"data_compressed_original":243919367,"omap_allocated":11150,"internal_metadata":279172210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":326353855,"num_objects":228,"num_object_clones":83,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":107,"num_read":17380,"num_read_kb":99795,"num_write":1662,"num_write_kb":51685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030584"},"pg_stats":[{"pgid":"3.1f","version":"59'206","reported_seq":2198,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.095427+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:22:02.095427+0000","last_peered":"2024-09-21T14:22:02.095427+0000","last_clean":"2024-09-21T14:22:02.095427+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:22:02.095427+0000","last_undegraded":"2024-09-21T14:22:02.095427+0000","last_fullsized":"2024-09-21T14:22:02.095427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":15177272,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2913,"num_read_kb":12085,"num_write":210,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'123","reported_seq":581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.673451+0000","last_change":"2024-09-21T14:21:49.388656+0000","last_active":"2024-09-21T14:22:01.673451+0000","last_peered":"2024-09-21T14:22:01.673451+0000","last_clean":"2024-09-21T14:22:01.673451+0000","last_became_active":"2024-09-21T14:21:49.343683+0000","last_became_peered":"2024-09-21T14:21:49.343683+0000","last_unstale":"2024-09-21T14:22:01.673451+0000","last_undegraded":"2024-09-21T14:22:01.673451+0000","last_fullsized":"2024-09-21T14:22:01.673451+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:13.419608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":25525634,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":397,"num_read_kb":9272,"num_write":115,"num_write_kb":5224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2693260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"59'138","reported_seq":1147,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.668006+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:22:01.668006+0000","last_peered":"2024-09-21T14:22:01.668006+0000","last_clean":"2024-09-21T14:22:01.668006+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:22:01.668006+0000","last_undegraded":"2024-09-21T14:22:01.668006+0000","last_fullsized":"2024-09-21T14:22:01.668006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":11488523,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":1036,"num_read_kb":8729,"num_write":165,"num_write_kb":4373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"60'190","reported_seq":3875,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.423059+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:22:02.423059+0000","last_peered":"2024-09-21T14:22:02.423059+0000","last_clean":"2024-09-21T14:22:02.423059+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:22:02.423059+0000","last_undegraded":"2024-09-21T14:22:02.423059+0000","last_fullsized":"2024-09-21T14:22:02.423059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":42682794,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":3787,"num_read_kb":31316,"num_write":190,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"59'221","reported_seq":2633,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.178330+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:22:02.178330+0000","last_peered":"2024-09-21T14:22:02.178330+0000","last_clean":"2024-09-21T14:22:02.178330+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:22:02.178330+0000","last_undegraded":"2024-09-21T14:22:02.178330+0000","last_fullsized":"2024-09-21T14:22:02.178330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":60930249,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":2793,"num_read_kb":28908,"num_write":192,"num_write_kb":12272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"60'228","reported_seq":2662,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.443191+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:22:02.443191+0000","last_peered":"2024-09-21T14:22:02.443191+0000","last_clean":"2024-09-21T14:22:02.443191+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:22:02.443191+0000","last_undegraded":"2024-09-21T14:22:02.443191+0000","last_fullsized":"2024-09-21T14:22:02.443191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":27181375,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":3490,"num_read_kb":5549,"num_write":270,"num_write_kb":2824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"60'177","reported_seq":4230,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.400438+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:22:02.400438+0000","last_peered":"2024-09-21T14:22:02.400438+0000","last_clean":"2024-09-21T14:22:02.400438+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:22:02.400438+0000","last_undegraded":"2024-09-21T14:22:02.400438+0000","last_fullsized":"2024-09-21T14:22:02.400438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":27472942,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":4566,"num_read_kb":21434,"num_write":165,"num_write_kb":2167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"57'118","reported_seq":901,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.668509+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:22:01.668509+0000","last_peered":"2024-09-21T14:22:01.668509+0000","last_clean":"2024-09-21T14:22:01.668509+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:22:01.668509+0000","last_undegraded":"2024-09-21T14:22:01.668509+0000","last_fullsized":"2024-09-21T14:22:01.668509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":11147374,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":809,"num_read_kb":2289,"num_write":106,"num_write_kb":1294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"59'392","reported_seq":1520,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.847425+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:22:01.847425+0000","last_peered":"2024-09-21T14:22:01.847425+0000","last_clean":"2024-09-21T14:22:01.847425+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:22:01.847425+0000","last_undegraded":"2024-09-21T14:22:01.847425+0000","last_fullsized":"2024-09-21T14:22:01.847425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":11195071,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":2753,"num_read_kb":6911,"num_write":533,"num_write_kb":3149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"57'137","reported_seq":1685,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.405412+0000","last_change":"2024-09-21T14:21:52.139246+0000","last_active":"2024-09-21T14:22:01.405412+0000","last_peered":"2024-09-21T14:22:01.405412+0000","last_clean":"2024-09-21T14:22:01.405412+0000","last_became_active":"2024-09-21T14:21:52.138595+0000","last_became_peered":"2024-09-21T14:21:52.138595+0000","last_unstale":"2024-09-21T14:22:01.405412+0000","last_undegraded":"2024-09-21T14:22:01.405412+0000","last_fullsized":"2024-09-21T14:22:01.405412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:23.662981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":7956855,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":1741,"num_read_kb":7561,"num_write":119,"num_write_kb":1251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":83845,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"58'413","reported_seq":1884,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.673724+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:22:01.673724+0000","last_peered":"2024-09-21T14:22:01.673724+0000","last_clean":"2024-09-21T14:22:01.673724+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:22:01.673724+0000","last_undegraded":"2024-09-21T14:22:01.673724+0000","last_fullsized":"2024-09-21T14:22:01.673724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":37470362,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":3305,"num_read_kb":21980,"num_write":560,"num_write_kb":8236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"59'250","reported_seq":1569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.178510+0000","last_change":"2024-09-21T14:21:52.140205+0000","last_active":"2024-09-21T14:22:02.178510+0000","last_peered":"2024-09-21T14:22:02.178510+0000","last_clean":"2024-09-21T14:22:02.178510+0000","last_became_active":"2024-09-21T14:21:52.138972+0000","last_became_peered":"2024-09-21T14:21:52.138972+0000","last_unstale":"2024-09-21T14:22:02.178510+0000","last_undegraded":"2024-09-21T14:22:02.178510+0000","last_fullsized":"2024-09-21T14:22:02.178510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:41.279903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":18588570,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":3059,"num_read_kb":3327,"num_write":302,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1075708,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.411818+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:22:01.411818+0000","last_peered":"2024-09-21T14:22:01.411818+0000","last_clean":"2024-09-21T14:22:01.411818+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:22:01.411818+0000","last_undegraded":"2024-09-21T14:22:01.411818+0000","last_fullsized":"2024-09-21T14:22:01.411818+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"59'203","reported_seq":1942,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.847176+0000","last_change":"2024-09-21T14:21:52.138940+0000","last_active":"2024-09-21T14:22:01.847176+0000","last_peered":"2024-09-21T14:22:01.847176+0000","last_clean":"2024-09-21T14:22:01.847176+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:22:01.847176+0000","last_undegraded":"2024-09-21T14:22:01.847176+0000","last_fullsized":"2024-09-21T14:22:01.847176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":38840074,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":20,"num_read":2265,"num_read_kb":6690,"num_write":189,"num_write_kb":2157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":59,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.405479+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:22:01.405479+0000","last_peered":"2024-09-21T14:22:01.405479+0000","last_clean":"2024-09-21T14:22:01.405479+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:22:01.405479+0000","last_undegraded":"2024-09-21T14:22:01.405479+0000","last_fullsized":"2024-09-21T14:22:01.405479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"59'322","reported_seq":1979,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.185009+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:22:02.185009+0000","last_peered":"2024-09-21T14:22:02.185009+0000","last_clean":"2024-09-21T14:22:02.185009+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:22:02.185009+0000","last_undegraded":"2024-09-21T14:22:02.185009+0000","last_fullsized":"2024-09-21T14:22:02.185009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":27183207,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":2978,"num_read_kb":5384,"num_write":460,"num_write_kb":2117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.400043+0000","last_change":"2024-09-21T14:21:49.338538+0000","last_active":"2024-09-21T14:22:01.400043+0000","last_peered":"2024-09-21T14:22:01.400043+0000","last_clean":"2024-09-21T14:22:01.400043+0000","last_became_active":"2024-09-21T14:21:49.338272+0000","last_became_peered":"2024-09-21T14:21:49.338272+0000","last_unstale":"2024-09-21T14:22:01.400043+0000","last_undegraded":"2024-09-21T14:22:01.400043+0000","last_fullsized":"2024-09-21T14:22:01.400043+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:55.449259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"59'176","reported_seq":1948,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.165889+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:22:02.165889+0000","last_peered":"2024-09-21T14:22:02.165889+0000","last_clean":"2024-09-21T14:22:02.165889+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:22:02.165889+0000","last_undegraded":"2024-09-21T14:22:02.165889+0000","last_fullsized":"2024-09-21T14:22:02.165889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":40851495,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":1776,"num_read_kb":28735,"num_write":152,"num_write_kb":7736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":59,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.485002+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:22:01.485002+0000","last_peered":"2024-09-21T14:22:01.485002+0000","last_clean":"2024-09-21T14:22:01.485002+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:22:01.485002+0000","last_undegraded":"2024-09-21T14:22:01.485002+0000","last_fullsized":"2024-09-21T14:22:01.485002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"59'174","reported_seq":4258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.203018+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:22:02.203018+0000","last_peered":"2024-09-21T14:22:02.203018+0000","last_clean":"2024-09-21T14:22:02.203018+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:22:02.203018+0000","last_undegraded":"2024-09-21T14:22:02.203018+0000","last_fullsized":"2024-09-21T14:22:02.203018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":42346199,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":4665,"num_read_kb":27355,"num_write":164,"num_write_kb":5666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"57'189","reported_seq":2158,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.650868+0000","last_change":"2024-09-21T14:21:52.139305+0000","last_active":"2024-09-21T14:22:01.650868+0000","last_peered":"2024-09-21T14:22:01.650868+0000","last_clean":"2024-09-21T14:22:01.650868+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:22:01.650868+0000","last_undegraded":"2024-09-21T14:22:01.650868+0000","last_fullsized":"2024-09-21T14:22:01.650868+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":36636910,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":2867,"num_read_kb":20255,"num_write":182,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":203,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.405225+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:22:01.405225+0000","last_peered":"2024-09-21T14:22:01.405225+0000","last_clean":"2024-09-21T14:22:01.405225+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:22:01.405225+0000","last_undegraded":"2024-09-21T14:22:01.405225+0000","last_fullsized":"2024-09-21T14:22:01.405225+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":60,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.403118+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:22:01.403118+0000","last_peered":"2024-09-21T14:22:01.403118+0000","last_clean":"2024-09-21T14:22:01.403118+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:22:01.403118+0000","last_undegraded":"2024-09-21T14:22:01.403118+0000","last_fullsized":"2024-09-21T14:22:01.403118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"59'447","reported_seq":1698,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.178135+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:22:02.178135+0000","last_peered":"2024-09-21T14:22:02.178135+0000","last_clean":"2024-09-21T14:22:02.178135+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:22:02.178135+0000","last_undegraded":"2024-09-21T14:22:02.178135+0000","last_fullsized":"2024-09-21T14:22:02.178135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":24563309,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2824,"num_read_kb":8059,"num_write":699,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":61,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.397602+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:22:01.397602+0000","last_peered":"2024-09-21T14:22:01.397602+0000","last_clean":"2024-09-21T14:22:01.397602+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:22:01.397602+0000","last_undegraded":"2024-09-21T14:22:01.397602+0000","last_fullsized":"2024-09-21T14:22:01.397602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"57'357","reported_seq":867,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.397918+0000","last_change":"2024-09-21T14:21:52.139498+0000","last_active":"2024-09-21T14:22:01.397918+0000","last_peered":"2024-09-21T14:22:01.397918+0000","last_clean":"2024-09-21T14:22:01.397918+0000","last_became_active":"2024-09-21T14:21:52.138744+0000","last_became_peered":"2024-09-21T14:21:52.138744+0000","last_unstale":"2024-09-21T14:22:01.397918+0000","last_undegraded":"2024-09-21T14:22:01.397918+0000","last_fullsized":"2024-09-21T14:22:01.397918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:16:08.802429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":17741403,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":1528,"num_read_kb":7785,"num_write":542,"num_write_kb":3890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2088730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.405321+0000","last_change":"2024-09-21T14:21:10.745649+0000","last_active":"2024-09-21T14:22:01.405321+0000","last_peered":"2024-09-21T14:22:01.405321+0000","last_clean":"2024-09-21T14:22:01.405321+0000","last_became_active":"2024-09-21T14:21:08.709296+0000","last_became_peered":"2024-09-21T14:21:08.709296+0000","last_unstale":"2024-09-21T14:22:01.405321+0000","last_undegraded":"2024-09-21T14:22:01.405321+0000","last_fullsized":"2024-09-21T14:22:01.405321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:15.211938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"59'420","reported_seq":1759,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.673891+0000","last_change":"2024-09-21T14:21:52.139320+0000","last_active":"2024-09-21T14:22:01.673891+0000","last_peered":"2024-09-21T14:22:01.673891+0000","last_clean":"2024-09-21T14:22:01.673891+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:22:01.673891+0000","last_undegraded":"2024-09-21T14:22:01.673891+0000","last_fullsized":"2024-09-21T14:22:01.673891+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":62534416,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":2381,"num_read_kb":9397,"num_write":557,"num_write_kb":4978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":59,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.411593+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:22:01.411593+0000","last_peered":"2024-09-21T14:22:01.411593+0000","last_clean":"2024-09-21T14:22:01.411593+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:22:01.411593+0000","last_undegraded":"2024-09-21T14:22:01.411593+0000","last_fullsized":"2024-09-21T14:22:01.411593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"60'441","reported_seq":2771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.428342+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:22:02.428342+0000","last_peered":"2024-09-21T14:22:02.428342+0000","last_clean":"2024-09-21T14:22:02.428342+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:22:02.428342+0000","last_undegraded":"2024-09-21T14:22:02.428342+0000","last_fullsized":"2024-09-21T14:22:02.428342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":53764636,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":5032,"num_read_kb":9949,"num_write":544,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"60'323","reported_seq":2290,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.438143+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:22:02.438143+0000","last_peered":"2024-09-21T14:22:02.438143+0000","last_clean":"2024-09-21T14:22:02.438143+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:22:02.438143+0000","last_undegraded":"2024-09-21T14:22:02.438143+0000","last_fullsized":"2024-09-21T14:22:02.438143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":70625345,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":3910,"num_read_kb":15232,"num_write":318,"num_write_kb":5421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"59'243","reported_seq":655,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.800127+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:22:01.800127+0000","last_peered":"2024-09-21T14:22:01.800127+0000","last_clean":"2024-09-21T14:22:01.800127+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:22:01.800127+0000","last_undegraded":"2024-09-21T14:22:01.800127+0000","last_fullsized":"2024-09-21T14:22:01.800127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":13318421,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":673,"num_read_kb":3163,"num_write":243,"num_write_kb":3563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'206","reported_seq":1460,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.484963+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:22:01.484963+0000","last_peered":"2024-09-21T14:22:01.484963+0000","last_clean":"2024-09-21T14:22:01.484963+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:22:01.484963+0000","last_undegraded":"2024-09-21T14:22:01.484963+0000","last_fullsized":"2024-09-21T14:22:01.484963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":33366655,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1640,"num_read_kb":6330,"num_write":346,"num_write_kb":3579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"60'338","reported_seq":930,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.408113+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:22:02.408113+0000","last_peered":"2024-09-21T14:22:02.408113+0000","last_clean":"2024-09-21T14:22:02.408113+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:22:02.408113+0000","last_undegraded":"2024-09-21T14:22:02.408113+0000","last_fullsized":"2024-09-21T14:22:02.408113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":21133974,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":17,"num_read":1855,"num_read_kb":5572,"num_write":433,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"57'183","reported_seq":1800,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.633758+0000","last_change":"2024-09-21T14:21:34.229265+0000","last_active":"2024-09-21T14:22:01.633758+0000","last_peered":"2024-09-21T14:22:01.633758+0000","last_clean":"2024-09-21T14:22:01.633758+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:22:01.633758+0000","last_undegraded":"2024-09-21T14:22:01.633758+0000","last_fullsized":"2024-09-21T14:22:01.633758+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:04.756321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":18872961,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":1880,"num_read_kb":10180,"num_write":169,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"59'781","reported_seq":2897,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.203197+0000","last_change":"2024-09-21T14:21:52.141637+0000","last_active":"2024-09-21T14:22:02.203197+0000","last_peered":"2024-09-21T14:22:02.203197+0000","last_clean":"2024-09-21T14:22:02.203197+0000","last_became_active":"2024-09-21T14:21:52.140890+0000","last_became_peered":"2024-09-21T14:21:52.140890+0000","last_unstale":"2024-09-21T14:22:02.203197+0000","last_undegraded":"2024-09-21T14:22:02.203197+0000","last_fullsized":"2024-09-21T14:22:02.203197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:12.353305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":56813925,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":5425,"num_read_kb":17189,"num_write":1268,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3052424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"58'212","reported_seq":2755,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.674275+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:22:01.674275+0000","last_peered":"2024-09-21T14:22:01.674275+0000","last_clean":"2024-09-21T14:22:01.674275+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:22:01.674275+0000","last_undegraded":"2024-09-21T14:22:01.674275+0000","last_fullsized":"2024-09-21T14:22:01.674275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":61395004,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":2867,"num_read_kb":13491,"num_write":200,"num_write_kb":16984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"58'220","reported_seq":2057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.411611+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:22:01.411611+0000","last_peered":"2024-09-21T14:22:01.411611+0000","last_clean":"2024-09-21T14:22:01.411611+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:22:01.411611+0000","last_undegraded":"2024-09-21T14:22:01.411611+0000","last_fullsized":"2024-09-21T14:22:01.411611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":36630188,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":2332,"num_read_kb":13172,"num_write":223,"num_write_kb":4045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"59'224","reported_seq":1556,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.057375+0000","last_change":"2024-09-21T14:21:52.139083+0000","last_active":"2024-09-21T14:22:02.057375+0000","last_peered":"2024-09-21T14:22:02.057375+0000","last_clean":"2024-09-21T14:22:02.057375+0000","last_became_active":"2024-09-21T14:21:52.138188+0000","last_became_peered":"2024-09-21T14:21:52.138188+0000","last_unstale":"2024-09-21T14:22:02.057375+0000","last_undegraded":"2024-09-21T14:22:02.057375+0000","last_fullsized":"2024-09-21T14:22:02.057375+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:15.976782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":55838171,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":1561,"num_read_kb":18840,"num_write":285,"num_write_kb":17241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":3452172,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"60'204","reported_seq":1788,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T14:22:02.427517+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:22:02.427517+0000","last_peered":"2024-09-21T14:22:02.427517+0000","last_clean":"2024-09-21T14:22:02.427517+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:22:02.427517+0000","last_undegraded":"2024-09-21T14:22:02.427517+0000","last_fullsized":"2024-09-21T14:22:02.427517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":26411215,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":2549,"num_read_kb":14589,"num_write":221,"num_write_kb":6765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"58'601","reported_seq":2953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:22:01.405294+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:22:01.405294+0000","last_peered":"2024-09-21T14:22:01.405294+0000","last_clean":"2024-09-21T14:22:01.405294+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:22:01.405294+0000","last_undegraded":"2024-09-21T14:22:01.405294+0000","last_fullsized":"2024-09-21T14:22:01.405294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":24156690,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":5587,"num_read_kb":19260,"num_write":878,"num_write_kb":9748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1059841219,"num_objects":1274,"num_object_clones":321,"num_object_copies":2548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1274,"num_whiteouts":528,"num_read":87244,"num_read_kb":419988,"num_write":11000,"num_write_kb":178522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31019008,"data_stored":267268417,"data_compressed":14973034,"data_compressed_allocated":17477632,"data_compressed_original":256371544,"omap_allocated":0,"internal_metadata":0},"log_size":8857,"ondisk_log_size":8857,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2724,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2789376,"data_stored":17379262,"data_compressed":899886,"data_compressed_allocated":1134592,"data_compressed_original":16329913,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39332,"kb_used_data":4120,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332508,"statfs":{"total":96636764160,"available":96596488192,"internally_reserved":0,"allocated":4218880,"data_stored":31819942,"data_compressed":1827867,"data_compressed_allocated":2093056,"data_compressed_original":30345342,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38596,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333244,"statfs":{"total":96636764160,"available":96597241856,"internally_reserved":0,"allocated":3461120,"data_stored":22167608,"data_compressed":1160951,"data_compressed_allocated":1495040,"data_compressed_original":20867884,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40896,"kb_used_data":5688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330944,"statfs":{"total":96636764160,"available":96594886656,"internally_reserved":0,"allocated":5824512,"data_stored":46797338,"data_compressed":2602437,"data_compressed_allocated":2850816,"data_compressed_original":44669268,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45408,"kb_used_data":6100,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326432,"statfs":{"total":96636764160,"available":96590266368,"internally_reserved":0,"allocated":6246400,"data_stored":46490474,"data_compressed":2630376,"data_compressed_allocated":3227648,"data_compressed_original":44158043,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":0,"seq":55834574861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40076,"kb_used_data":4856,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331764,"statfs":{"total":96636764160,"available":96595726336,"internally_reserved":0,"allocated":4972544,"data_stored":35350979,"data_compressed":2030675,"data_compressed_allocated":2363392,"data_compressed_original":33459277,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63844,"kb_used_data":6492,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307996,"statfs":{"total":96636764160,"available":96571387904,"internally_reserved":0,"allocated":6647808,"data_stored":56483974,"data_compressed":3126116,"data_compressed_allocated":3362816,"data_compressed_original":54089640,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5799936,"data_stored":55258747,"data_compressed":3068163,"data_compressed_allocated":3284992,"data_compressed_original":53245864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4091904,"data_stored":34357949,"data_compressed":1970047,"data_compressed_allocated":2285568,"data_compressed_original":32885837,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5406720,"data_stored":45032243,"data_compressed":2555195,"data_compressed_allocated":3129344,"data_compressed_original":43060315,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5148672,"data_stored":46488502,"data_compressed":2602437,"data_compressed_allocated":2850816,"data_compressed_original":44669268,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2945024,"data_stored":22709812,"data_compressed":1226259,"data_compressed_allocated":1572864,"data_compressed_original":21649351,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3497984,"data_stored":31015716,"data_compressed":1798588,"data_compressed_allocated":2052096,"data_compressed_original":29849726,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2129920,"data_stored":16330733,"data_compressed":864179,"data_compressed_allocated":1089536,"data_compressed_original":15530324,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:22:06.211 INFO:tasks.thrashosds.thrasher:Setting 1.1 to [5, 4] 2024-09-21T14:22:06.211 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.1', '5', '4'] 2024-09-21T14:22:06.211 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.1 5 4 2024-09-21T14:22:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:06.282 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:06.300 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:06.301+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:06.401 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:06.401+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:06.431 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.888106835999999, 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43094 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042580671, 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355065000000002, 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43094 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387470+0000", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5480000000000003e-06 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387475+0000", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1560000000000003e-06 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387479+0000", 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7330000000000001e-06 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427814+0000", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040335226000000002 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427821+0000", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6889999999999998e-06 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43095 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042563896000001, 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040366234000000001, 2024-09-21T14:22:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43095 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387484+0000", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.857e-06 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387489+0000", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387493+0000", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2830000000000003e-06 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427842+0000", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040348342000000002 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427849+0000", 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9759999999999998e-06 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042549343000001, 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:22:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042530473999999, 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042516442, 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042500443000002, 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042485546999998, 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042470610999999, 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042455630999999, 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042441015000001, 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042425968, 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042411409, 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042396684, 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042382243999999, 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042367894000002, 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042347828, 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042332918, 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042317978, 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.042298285000001, 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:06.509 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:06.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:06.511 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone32 from 0xd37248b (overlap 0x216d7d9) to 0x357bfc 2024-09-21T14:22:06.511 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:06.510+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:06.550 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T14:22:06.565 INFO:teuthology.orchestra.run.smithi073.stdout:3650 trunc from 0xd37248b to 0x1bd8f63 2024-09-21T14:22:06.611 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:06.611+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:06.628 INFO:teuthology.orchestra.run.smithi073.stdout:3652 write 0x1c1e51b thru 0x1c20ee1 (0x29c7 bytes) 2024-09-21T14:22:06.632 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:06.632 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:06.632 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:06.632 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:06.637 INFO:teuthology.orchestra.run.smithi073.stdout:3653 writesame 0x4267c9d thru 0x427429e data_size 0x25(0xc602 bytes) 2024-09-21T14:22:06.637 INFO:teuthology.orchestra.run.smithi073.stdout:3654 punch from 0x411b91b to 0x4121c20, (0x6305 bytes) 2024-09-21T14:22:06.638 INFO:teuthology.orchestra.run.smithi073.stdout:3656 read 0xf922ea thru 0xfa00bf (0xddd6 bytes) 2024-09-21T14:22:06.638 INFO:teuthology.orchestra.run.smithi073.stdout:3657 writesame 0x33929b6 thru 0x339864e data_size 0xb(0x5c99 bytes) 2024-09-21T14:22:06.640 INFO:teuthology.orchestra.run.smithi073.stdout:3661 read 0x2654a41 thru 0x2663291 (0xe851 bytes) 2024-09-21T14:22:06.640 INFO:teuthology.orchestra.run.smithi073.stdout:3662 write 0x2c5e27 thru 0x2ce74d (0x8927 bytes) 2024-09-21T14:22:06.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:06.648 INFO:teuthology.orchestra.run.smithi073.stdout:3663 write 0x58335ae thru 0x583d79b (0xa1ee bytes) 2024-09-21T14:22:06.667 INFO:teuthology.orchestra.run.smithi073.stdout:3665 write 0xe87d975 thru 0xe884354 (0x69e0 bytes) 2024-09-21T14:22:06.672 INFO:teuthology.orchestra.run.smithi073.stdout:3666 writesame 0x5bb57ce thru 0x5bc505c data_size 0x13(0xf88f bytes) 2024-09-21T14:22:06.675 INFO:teuthology.orchestra.run.smithi073.stdout:3668 writesame 0xdefe5c thru 0xdff595 data_size 0x5(0xf73a bytes) 2024-09-21T14:22:06.682 INFO:teuthology.orchestra.run.smithi073.stdout:3669 write 0xedc72f thru 0xedfee5 (0x37b7 bytes) 2024-09-21T14:22:06.684 INFO:teuthology.orchestra.run.smithi073.stdout:3672 punch from 0x2039939 to 0x204919f, (0xf866 bytes) 2024-09-21T14:22:06.684 INFO:teuthology.orchestra.run.smithi073.stdout:3674 write 0xc226055 thru 0xc233ed6 (0xde82 bytes) 2024-09-21T14:22:06.687 INFO:teuthology.orchestra.run.smithi073.stdout:3675 read 0x3cb45e thru 0x3cf692 (0x4235 bytes) 2024-09-21T14:22:06.703 INFO:teuthology.orchestra.run.smithi073.stdout:3676 trunc from 0xe884355 to 0x16f7f1d 2024-09-21T14:22:06.712 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:06.713+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:06.718 INFO:teuthology.orchestra.run.smithi073.stdout:3677 read 0x7ba777 thru 0x7c7935 (0xd1bf bytes) 2024-09-21T14:22:06.719 INFO:teuthology.orchestra.run.smithi073.stdout:3678 write 0x24d239 thru 0x251b3b (0x4903 bytes) 2024-09-21T14:22:06.720 INFO:teuthology.orchestra.run.smithi073.stdout:3679 read 0x652edd thru 0x65b550 (0x8674 bytes) 2024-09-21T14:22:06.738 INFO:teuthology.orchestra.run.smithi073.stdout:3680 write 0xc5bbba1 thru 0xc5c6043 (0xa4a3 bytes) 2024-09-21T14:22:06.743 INFO:teuthology.orchestra.run.smithi073.stdout:3681 writesame 0x33fb672 thru 0x3403036 data_size 0x1(0x79c5 bytes) 2024-09-21T14:22:06.746 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:06.746 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:06.746 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:06.746 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:06.746 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:06.752 INFO:teuthology.orchestra.run.smithi073.stdout:3682 read 0x557b0e0 thru 0x5588472 (0xd393 bytes) 2024-09-21T14:22:06.752 INFO:teuthology.orchestra.run.smithi073.stdout:3683 read 0x938057 thru 0x943404 (0xb3ae bytes) 2024-09-21T14:22:06.754 INFO:teuthology.orchestra.run.smithi073.stdout:3684 read 0x826dd91 thru 0x826efbb (0x122b bytes) 2024-09-21T14:22:06.754 INFO:teuthology.orchestra.run.smithi073.stdout:3685 read 0x57aff78 thru 0x57ba51a (0xa5a3 bytes) 2024-09-21T14:22:06.754 INFO:teuthology.orchestra.run.smithi073.stdout:3686 write 0xbec69c3 thru 0xbed475b (0xdd99 bytes) 2024-09-21T14:22:06.755 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:06.756 INFO:teuthology.orchestra.run.smithi073.stdout:3687 writesame 0x1e4615f thru 0x1e54a8b data_size 0x1(0xe92d bytes) 2024-09-21T14:22:06.776 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.776 INFO:teuthology.orchestra.run.smithi073.stdout:3690 read 0x3ece472 thru 0x3ed752f (0x90be bytes) 2024-09-21T14:22:06.776 INFO:teuthology.orchestra.run.smithi073.stdout:3691 write 0x1034c71 thru 0x1039ddb (0x516b bytes) 2024-09-21T14:22:06.779 INFO:teuthology.orchestra.run.smithi073.stdout:3693 write 0xb2ad424 thru 0xb2b2358 (0x4f35 bytes) 2024-09-21T14:22:06.782 INFO:teuthology.orchestra.run.smithi073.stdout:3694 write 0x97e82bc thru 0x97f45d7 (0xc31c bytes) 2024-09-21T14:22:06.785 INFO:teuthology.orchestra.run.smithi073.stdout:3698 write 0x3dd2015 thru 0x3dda4c2 (0x84ae bytes) 2024-09-21T14:22:06.788 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.788 INFO:teuthology.orchestra.run.smithi073.stdout:3700 write 0x4921b07 thru 0x492ee47 (0xd341 bytes) 2024-09-21T14:22:06.790 INFO:teuthology.orchestra.run.smithi073.stdout:3701 read 0xa8f073e thru 0xa8faeca (0xa78d bytes) 2024-09-21T14:22:06.790 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.790 INFO:teuthology.orchestra.run.smithi073.stdout:3707 write 0x4d91ed3 thru 0x4d9829c (0x63ca bytes) 2024-09-21T14:22:06.791 INFO:teuthology.orchestra.run.smithi073.stdout:3709 read 0x16cd674 thru 0x16d882f (0xb1bc bytes) 2024-09-21T14:22:06.792 INFO:teuthology.orchestra.run.smithi073.stdout:3710 read 0x2e6980d thru 0x2e6bd6f (0x2563 bytes) 2024-09-21T14:22:06.792 INFO:teuthology.orchestra.run.smithi073.stdout:3711 writesame 0x5c7d235 thru 0x5c8af46 data_size 0x1(0xdd12 bytes) 2024-09-21T14:22:06.814 INFO:teuthology.orchestra.run.smithi073.stdout:3713 read 0x906bfd4 thru 0x90782b9 (0xc2e6 bytes) 2024-09-21T14:22:06.816 INFO:teuthology.orchestra.run.smithi073.stdout:3714 trunc from 0xc5c6044 to 0xe90c071 2024-09-21T14:22:06.821 INFO:teuthology.orchestra.run.smithi073.stdout:3715 write 0xa85a13f thru 0xa85c090 (0x1f52 bytes) 2024-09-21T14:22:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:3716 trunc from 0xe90c071 to 0xaf76204 2024-09-21T14:22:06.837 INFO:teuthology.orchestra.run.smithi073.stdout:3717 trunc from 0xaf76204 to 0xd48be77 2024-09-21T14:22:06.840 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:06.840+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:06.842 INFO:teuthology.orchestra.run.smithi073.stdout:3719 write 0x678a01d thru 0x6799af0 (0xfad4 bytes) 2024-09-21T14:22:06.845 INFO:teuthology.orchestra.run.smithi073.stdout:3720 read 0xbe61d39 thru 0xbe69ffc (0x82c4 bytes) 2024-09-21T14:22:06.845 INFO:teuthology.orchestra.run.smithi073.stdout:3721 writesame 0x1e18934 thru 0x1e1a23a data_size 0x1(0x1907 bytes) 2024-09-21T14:22:06.848 INFO:teuthology.orchestra.run.smithi073.stdout:3722 writesame 0xbc134ce thru 0xbc1b174 data_size 0xb(0x7ca7 bytes) 2024-09-21T14:22:06.851 INFO:teuthology.orchestra.run.smithi073.stdout:3723 write 0x762c2d9 thru 0x7638648 (0xc370 bytes) 2024-09-21T14:22:06.853 INFO:teuthology.orchestra.run.smithi073.stdout:3724 write 0xa350768 thru 0xa35e10b (0xd9a4 bytes) 2024-09-21T14:22:06.855 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.869 INFO:teuthology.orchestra.run.smithi073.stdout:3729 trunc from 0xd48be77 to 0x18bebe8 2024-09-21T14:22:06.881 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.894 INFO:teuthology.orchestra.run.smithi073.stdout:3734 writesame 0x8e815df thru 0x8e8faf4 data_size 0xe(0xe516 bytes) 2024-09-21T14:22:06.895 INFO:teuthology.orchestra.run.smithi073.stdout:3735 write 0x487fb4e thru 0x488d62b (0xdade bytes) 2024-09-21T14:22:06.898 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.898 INFO:teuthology.orchestra.run.smithi073.stdout:3737 read 0x266c2 thru 0x297cd (0x310c bytes) 2024-09-21T14:22:06.898 INFO:teuthology.orchestra.run.smithi073.stdout:3740 punch from 0x2ace850 to 0x2adcc15, (0xe3c5 bytes) 2024-09-21T14:22:06.904 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:06.904 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:06.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:06.904 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:06.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43741 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513855226, 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036952564, 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43741 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390192+0000", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.14e-06 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390199+0000", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5150000000000003e-06 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390204+0000", 2024-09-21T14:22:06.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000001e-06 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.426948+0000", 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036744128000000001 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427034+0000", 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6402000000000004e-05 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427137+0000", 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010275799999999999 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43742 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513830419000001, 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037087785999999998, 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.909 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43742 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390211+0000", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1019999999999999e-06 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390216+0000", 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2870000000000004e-06 2024-09-21T14:22:06.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390221+0000", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427168+0000", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036947520999999997 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427213+0000", 2024-09-21T14:22:06.912 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4718000000000001e-05 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427297+0000", 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3939999999999999e-05 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43743 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513814867000001, 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037293508000000003, 2024-09-21T14:22:06.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43743 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.914 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390224+0000", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390227+0000", 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:22:06.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390232+0000", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5799999999999999e-06 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390237+0000", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7890000000000002e-06 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427333+0000", 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037095876999999999 2024-09-21T14:22:06.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6895000000000001e-05 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427518+0000", 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8611999999999997e-05 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.917 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43744 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513798677, 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037409745000000001, 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43744 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.918 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390243+0000", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390249+0000", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390253+0000", 2024-09-21T14:22:06.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7829999999999998e-06 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427550+0000", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037296636000000001 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427608+0000", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8100000000000003e-05 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427651+0000", 2024-09-21T14:22:06.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2528999999999999e-05 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43745 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513782367000001, 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037596321000000002, 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.922 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43745 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390259+0000", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0770000000000002e-06 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390264+0000", 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3430000000000003e-06 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390268+0000", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0470000000000004e-06 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427691+0000", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037422596000000002 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427784+0000", 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2780000000000003e-05 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427854+0000", 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9770000000000005e-05 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43746 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513768471999999, 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037742629, 2024-09-21T14:22:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43746 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.927 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390273+0000", 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1150000000000001e-06 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.928 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390279+0000", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6799999999999998e-06 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390283+0000", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427891+0000", 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037607640999999997 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427973+0000", 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2170999999999996e-05 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428014+0000", 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0806000000000002e-05 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.930 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43747 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513720081999999, 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037837335999999999, 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43747 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.931 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390319+0000", 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.932 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390323+0000", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390332+0000", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9600000000000006e-06 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390339+0000", 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8959999999999997e-06 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428048+0000", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037709135999999997 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428117+0000", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8534999999999993e-05 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428157+0000", 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0117000000000002e-05 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.934 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43748 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513695304999999, 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037945300000000001, 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.935 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43748 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390346+0000", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.464e-06 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390354+0000", 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9619999999999993e-06 2024-09-21T14:22:06.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390359+0000", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428184+0000", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037824759999999999 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428230+0000", 2024-09-21T14:22:06.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6112000000000002e-05 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428290+0000", 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9327000000000003e-05 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43749 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513675609, 2024-09-21T14:22:06.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038038324999999998, 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43749 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390368+0000", 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1579999999999998e-06 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390375+0000", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390380+0000", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9609999999999996e-06 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428319+0000", 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037939925999999999 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428364+0000", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4623999999999998e-05 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428402+0000", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43752 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.51365582, 2024-09-21T14:22:06.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038243006000000003, 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43752 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390383+0000", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390386+0000", 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0640000000000001e-06 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390392+0000", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4269999999999996e-06 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390397+0000", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7650000000000001e-06 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428433+0000", 2024-09-21T14:22:06.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038036164999999997 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428525+0000", 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1831999999999998e-05 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428627+0000", 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102099 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43753 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513638577000002, 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038351382000000003, 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43753 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390403+0000", 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0990000000000001e-06 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390410+0000", 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7299999999999999e-06 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390415+0000", 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0730000000000004e-06 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428653+0000", 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038237924 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428697+0000", 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4367999999999999e-05 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428752+0000", 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5535000000000003e-05 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43791 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513620393, 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038564714, 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.952 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43791 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:06.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390421+0000", 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.345e-06 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390428+0000", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6669999999999999e-06 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390433+0000", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.169e-06 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428791+0000", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038358033 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428890+0000", 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9216999999999996e-05 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428984+0000", 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3666999999999996e-05 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43792 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513602227, 2024-09-21T14:22:06.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038678535, 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43792 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390444+0000", 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390451+0000", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390460+0000", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9150000000000008e-06 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429008+0000", 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038547887000000003 2024-09-21T14:22:06.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429052+0000", 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3509e-05 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429116+0000", 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4078999999999994e-05 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43802 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513573718, 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038886423000000003, 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43802 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:06.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390468+0000", 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.328e-06 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390476+0000", 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1939999999999998e-06 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390481+0000", 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2449999999999998e-06 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429158+0000", 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038676675000000001 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429249+0000", 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0657999999999998e-05 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429352+0000", 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103729 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43803 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513553860999998, 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039086681999999998, 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.965 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43803 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390485+0000", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390488+0000", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1789999999999998e-06 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390494+0000", 2024-09-21T14:22:06.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3040000000000001e-06 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390499+0000", 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429393+0000", 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038894038999999998 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429487+0000", 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4504000000000002e-05 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429572+0000", 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.509e-05 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43819 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513536523999999, 2024-09-21T14:22:06.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039269071000000003, 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43819 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390505+0000", 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1490000000000001e-06 2024-09-21T14:22:06.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390511+0000", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0290000000000002e-06 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390516+0000", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8060000000000002e-06 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429608+0000", 2024-09-21T14:22:06.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039092162 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429697+0000", 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8712e-05 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429772+0000", 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5586000000000004e-05 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.973 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43820 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513293772000001, 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039230445000000003, 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43820 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390745+0000", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390749+0000", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.799e-06 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390770+0000", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1035000000000001e-05 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390777+0000", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9890000000000003e-06 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:22:06.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039033209999999999 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429904+0000", 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4094999999999999e-05 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429976+0000", 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1931e-05 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43832 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513256943999998, 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039328466999999999, 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43832 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390782+0000", 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390785+0000", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8870000000000001e-06 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390792+0000", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.951e-06 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390797+0000", 2024-09-21T14:22:06.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1780000000000002e-06 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430003+0000", 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039205885000000003 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430046+0000", 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2867000000000001e-05 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430111+0000", 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5242999999999999e-05 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43833 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513237982, 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039492185999999999, 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43833 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390801+0000", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390804+0000", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390810+0000", 2024-09-21T14:22:06.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4999999999999996e-06 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390815+0000", 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430137+0000", 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039321964000000001 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430192+0000", 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4787000000000002e-05 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430294+0000", 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102186 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43845 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 151552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.513220260000001, 2024-09-21T14:22:06.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039630430000000001, 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43845 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390821+0000", 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2689999999999998e-06 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390833+0000", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2248e-05 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390838+0000", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9540000000000003e-06 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430335+0000", 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039496297999999999 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430389+0000", 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4293999999999999e-05 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430450+0000", 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0757999999999997e-05 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:06.990 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:06.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3741 write 0xe107acb thru 0xe113d12 (0xc248 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3742 punch from 0x8f14b88 to 0x8f1eccc, (0xa144 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3743 read 0xcf1b56b thru 0xcf284dd (0xcf73 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3744 read 0xbb22fe7 thru 0xbb30827 (0xd841 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3745 write 0xc6afd18 thru 0xc6b741b (0x7704 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3746 write 0x5bf57e4 thru 0x5bfada8 (0x55c5 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3748 read 0xa2e7be8 thru 0xa2ea48f (0x28a8 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3749 punch from 0xc02477 to 0xc07824, (0x53ad bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3750 trunc from 0xe113d13 to 0x2fedfed 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3754 trunc from 0x2fedfed to 0x71de5be 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3755 read 0x16f5932 thru 0x1700b27 (0xb1f6 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3760 write 0xe004dc9 thru 0xe01460d (0xf845 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:3761 read 0x7df8db3 thru 0x7e055e2 (0xc830 bytes) 2024-09-21T14:22:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:3763 writesame 0x985c06b thru 0x985d397 data_size 0x1(0x132d bytes) 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:3764 trunc from 0xe01460e to 0xad10676 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:3765 writesame 0x243e23a thru 0x2441772 data_size 0x5(0x3539 bytes) 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:3767 punch from 0x1e76347 to 0x1e7bdfd, (0x5ab6 bytes) 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:3768 write 0x489d516 thru 0x48a9cdb (0xc7c6 bytes) 2024-09-21T14:22:06.995 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:06.991+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:06.996 INFO:teuthology.orchestra.run.smithi073.stdout:3770 trunc from 0xad10676 to 0xc9f2d43 2024-09-21T14:22:06.999 INFO:teuthology.orchestra.run.smithi073.stdout:3771 write 0xb049c2f thru 0xb051065 (0x7437 bytes) 2024-09-21T14:22:07.002 INFO:teuthology.orchestra.run.smithi073.stdout:3772 read 0x35d9be5 thru 0x35e3c57 (0xa073 bytes) 2024-09-21T14:22:07.003 INFO:teuthology.orchestra.run.smithi073.stdout:3773 write 0x140d331 thru 0x141a5d4 (0xd2a4 bytes) 2024-09-21T14:22:07.005 INFO:teuthology.orchestra.run.smithi073.stdout:3775 read 0x75e782f thru 0x75ec32e (0x4b00 bytes) 2024-09-21T14:22:07.005 INFO:teuthology.orchestra.run.smithi073.stdout:3776 write 0xb7e98b2 thru 0xb7f78c9 (0xe018 bytes) 2024-09-21T14:22:07.007 INFO:teuthology.orchestra.run.smithi073.stdout:3777 write 0x614f4c4 thru 0x6153c30 (0x476d bytes) 2024-09-21T14:22:07.010 INFO:teuthology.orchestra.run.smithi073.stdout:3778 write 0x5f12201 thru 0x5f21f8a (0xfd8a bytes) 2024-09-21T14:22:07.017 INFO:teuthology.orchestra.run.smithi073.stdout:3779 writesame 0x13c1b5c thru 0x13c6860 data_size 0x1(0x4d05 bytes) 2024-09-21T14:22:07.026 INFO:teuthology.orchestra.run.smithi073.stdout:3780 write 0x3838761 thru 0x383fa28 (0x72c8 bytes) 2024-09-21T14:22:07.029 INFO:teuthology.orchestra.run.smithi073.stdout:3783 read 0x830f4ba thru 0x831e590 (0xf0d7 bytes) 2024-09-21T14:22:07.036 INFO:teuthology.orchestra.run.smithi073.stdout:3784 write 0xdf4e48f thru 0xdf55ee5 (0x7a57 bytes) 2024-09-21T14:22:07.038 INFO:teuthology.orchestra.run.smithi073.stdout:3785 write 0x5fea51f thru 0x5fed985 (0x3467 bytes) 2024-09-21T14:22:07.040 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.040 INFO:teuthology.orchestra.run.smithi073.stdout:3787 write 0x31e69cc thru 0x31f0057 (0x968c bytes) 2024-09-21T14:22:07.045 INFO:teuthology.orchestra.run.smithi073.stdout:3788 trunc from 0xdf55ee6 to 0xaffbd96 2024-09-21T14:22:07.051 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.061 INFO:teuthology.orchestra.run.smithi073.stdout:3790 trunc from 0xaffbd96 to 0x28ca36b 2024-09-21T14:22:07.080 INFO:teuthology.orchestra.run.smithi073.stdout:3791 write 0x2e3d378 thru 0x2e4289e (0x5527 bytes) 2024-09-21T14:22:07.087 INFO:teuthology.orchestra.run.smithi073.stdout:3792 write 0x38ce125 thru 0x38da7cf (0xc6ab bytes) 2024-09-21T14:22:07.092 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:07.092+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:07.103 INFO:teuthology.orchestra.run.smithi073.stdout:3793 write 0x8020266 thru 0x8024b0e (0x48a9 bytes) 2024-09-21T14:22:07.109 INFO:teuthology.orchestra.run.smithi073.stdout:3794 punch from 0x74a06f2 to 0x74a7061, (0x696f bytes) 2024-09-21T14:22:07.109 INFO:teuthology.orchestra.run.smithi073.stdout:3796 read 0x2abbeb4 thru 0x2acb65e (0xf7ab bytes) 2024-09-21T14:22:07.109 INFO:teuthology.orchestra.run.smithi073.stdout:3797 write 0x411ab31 thru 0x411dbdf (0x30af bytes) 2024-09-21T14:22:07.114 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.114 INFO:teuthology.orchestra.run.smithi073.stdout:3800 read 0x1895d01 thru 0x1897f5f (0x225f bytes) 2024-09-21T14:22:07.114 INFO:teuthology.orchestra.run.smithi073.stdout:3801 read 0x199204 thru 0x19fb17 (0x6914 bytes) 2024-09-21T14:22:07.117 INFO:teuthology.orchestra.run.smithi073.stdout:3802 trunc from 0x8024b0f to 0xa2bf0e5 2024-09-21T14:22:07.122 INFO:teuthology.orchestra.run.smithi073.stdout:3803 write 0x7f7be2f thru 0x7f7f1ba (0x338c bytes) 2024-09-21T14:22:07.125 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.125 INFO:teuthology.orchestra.run.smithi073.stdout:3806 read 0x174ee91 thru 0x17531b2 (0x4322 bytes) 2024-09-21T14:22:07.125 INFO:teuthology.orchestra.run.smithi073.stdout:3807 write 0xf2e0d2 thru 0xf3a147 (0xc076 bytes) 2024-09-21T14:22:07.127 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.127 INFO:teuthology.orchestra.run.smithi073.stdout:3809 punch from 0x352c043 to 0x352c685, (0x642 bytes) 2024-09-21T14:22:07.127 INFO:teuthology.orchestra.run.smithi073.stdout:3810 write 0x19677ec thru 0x1973790 (0xbfa5 bytes) 2024-09-21T14:22:07.128 INFO:teuthology.orchestra.run.smithi073.stdout:3811 read 0x9e6e2d4 thru 0x9e72206 (0x3f33 bytes) 2024-09-21T14:22:07.129 INFO:teuthology.orchestra.run.smithi073.stdout:3812 read 0x1a5b74c thru 0x1a5d7f8 (0x20ad bytes) 2024-09-21T14:22:07.129 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.129 INFO:teuthology.orchestra.run.smithi073.stdout:3814 write 0x909a96e thru 0x90a72c7 (0xc95a bytes) 2024-09-21T14:22:07.132 INFO:teuthology.orchestra.run.smithi073.stdout:3815 read 0x13a6e3b thru 0x13b2103 (0xb2c9 bytes) 2024-09-21T14:22:07.133 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.133 INFO:teuthology.orchestra.run.smithi073.stdout:3817 write 0x8428299 thru 0x8428e70 (0xbd8 bytes) 2024-09-21T14:22:07.138 INFO:teuthology.orchestra.run.smithi073.stdout:3818 trunc from 0xa2bf0e5 to 0x807592d 2024-09-21T14:22:07.141 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:07.141 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:07.141 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:07.142 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:07.144 INFO:teuthology.orchestra.run.smithi073.stdout:3819 punch from 0x311e0ae to 0x31257bd, (0x770f bytes) 2024-09-21T14:22:07.150 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:07.151 INFO:teuthology.orchestra.run.smithi073.stdout:3820 trunc from 0x807592d to 0x23bee10 2024-09-21T14:22:07.163 INFO:teuthology.orchestra.run.smithi073.stdout:3821 write 0xe765cd thru 0xe834fb (0xcf2f bytes) 2024-09-21T14:22:07.165 INFO:teuthology.orchestra.run.smithi073.stdout:3822 write 0x166982c thru 0x1677e40 (0xe615 bytes) 2024-09-21T14:22:07.176 INFO:teuthology.orchestra.run.smithi073.stdout:3823 trunc from 0x23bee10 to 0xac561ec 2024-09-21T14:22:07.185 INFO:teuthology.orchestra.run.smithi073.stdout:3824 trunc from 0xac561ec to 0x7a3fbf6 2024-09-21T14:22:07.193 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:07.194+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:07.195 INFO:teuthology.orchestra.run.smithi073.stdout:3827 trunc from 0x7a3fbf6 to 0xb14fa84 2024-09-21T14:22:07.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:07.200 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.200 INFO:teuthology.orchestra.run.smithi073.stdout:3829 punch from 0x87dbf1e to 0x87de4a4, (0x2586 bytes) 2024-09-21T14:22:07.203 INFO:teuthology.orchestra.run.smithi073.stdout:3830 trunc from 0xb14fa84 to 0xd609058 2024-09-21T14:22:07.208 INFO:teuthology.orchestra.run.smithi073.stdout:3831 read 0x5341290 thru 0x534857b (0x72ec bytes) 2024-09-21T14:22:07.209 INFO:teuthology.orchestra.run.smithi073.stdout:3833 trunc from 0xd609058 to 0xe2bf9e7 2024-09-21T14:22:07.214 INFO:teuthology.orchestra.run.smithi073.stdout:3835 write 0x610fe23 thru 0x611c3c3 (0xc5a1 bytes) 2024-09-21T14:22:07.222 INFO:teuthology.orchestra.run.smithi073.stdout:3836 write 0xe5cdf5f thru 0xe5d40cb (0x616d bytes) 2024-09-21T14:22:07.226 INFO:teuthology.orchestra.run.smithi073.stdout:3837 read 0xd183785 thru 0xd1872bd (0x3b39 bytes) 2024-09-21T14:22:07.226 INFO:teuthology.orchestra.run.smithi073.stdout:3838 write 0x58746b0 thru 0x587a836 (0x6187 bytes) 2024-09-21T14:22:07.229 INFO:teuthology.orchestra.run.smithi073.stdout:3839 read 0x99b45a7 thru 0x99c07ec (0xc246 bytes) 2024-09-21T14:22:07.229 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.229 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.229 INFO:teuthology.orchestra.run.smithi073.stdout:3842 write 0x9d0266a thru 0x9d04ecd (0x2864 bytes) 2024-09-21T14:22:07.248 INFO:teuthology.orchestra.run.smithi073.stdout:3844 trunc from 0xe5d40cc to 0xc95b64 2024-09-21T14:22:07.258 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:07.258 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:07.258 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:07.258 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:07.258 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:07.260 INFO:teuthology.orchestra.run.smithi073.stdout:3845 read 0x2ff151 thru 0x303c3e (0x4aee bytes) 2024-09-21T14:22:07.267 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:07.281 INFO:teuthology.orchestra.run.smithi073.stdout:3846 write 0xe1ca96f thru 0xe1d56b6 (0xad48 bytes) 2024-09-21T14:22:07.284 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.284 INFO:teuthology.orchestra.run.smithi073.stdout:3849 writesame 0x5b4b982 thru 0x5b595c5 data_size 0x25(0xdc44 bytes) 2024-09-21T14:22:07.287 INFO:teuthology.orchestra.run.smithi073.stdout:3851 read 0xa17f2cb thru 0xa185606 (0x633c bytes) 2024-09-21T14:22:07.288 INFO:teuthology.orchestra.run.smithi073.stdout:3852 punch from 0xae4c581 to 0xae50940, (0x43bf bytes) 2024-09-21T14:22:07.288 INFO:teuthology.orchestra.run.smithi073.stdout:3854 read 0x2e92ee8 thru 0x2e9c521 (0x963a bytes) 2024-09-21T14:22:07.288 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.288 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.293 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:07.293+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:07.295 INFO:teuthology.orchestra.run.smithi073.stdout:3860 trunc from 0xe1d56b7 to 0x7e9de88 2024-09-21T14:22:07.304 INFO:teuthology.orchestra.run.smithi073.stdout:3861 writesame 0x499284c thru 0x499f9d6 data_size 0x1(0xd18b bytes) 2024-09-21T14:22:07.330 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:07.330 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:07.331 INFO:teuthology.orchestra.run.smithi073.stdout:3863 read 0x15085bf thru 0x150a0e9 (0x1b2b bytes) 2024-09-21T14:22:07.331 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.331 INFO:teuthology.orchestra.run.smithi073.stdout:3866 writesame 0x3ca3013 thru 0x3cb21bf data_size 0x29(0xf1ad bytes) 2024-09-21T14:22:07.333 INFO:teuthology.orchestra.run.smithi073.stdout:3868 punch from 0x1d6dea4 to 0x1d74e24, (0x6f80 bytes) 2024-09-21T14:22:07.333 INFO:teuthology.orchestra.run.smithi073.stdout:3869 write 0x2f18a6a thru 0x2f2006b (0x7602 bytes) 2024-09-21T14:22:07.334 INFO:teuthology.orchestra.run.smithi073.stdout:3870 read 0x24d4ef5 thru 0x24dba45 (0x6b51 bytes) 2024-09-21T14:22:07.335 INFO:teuthology.orchestra.run.smithi073.stdout:3871 writesame 0x7bd27a2 thru 0x7bda25e data_size 0x1(0x7abd bytes) 2024-09-21T14:22:07.348 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:07.348 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:07.352 INFO:teuthology.orchestra.run.smithi073.stdout:3872 read 0x452292 thru 0x45b65d (0x93cc bytes) 2024-09-21T14:22:07.361 INFO:teuthology.orchestra.run.smithi073.stdout:3873 write 0x9f03f25 thru 0x9f04769 (0x845 bytes) 2024-09-21T14:22:07.364 INFO:teuthology.orchestra.run.smithi073.stdout:3874 read 0x4d95d87 thru 0x4da3e86 (0xe100 bytes) 2024-09-21T14:22:07.364 INFO:teuthology.orchestra.run.smithi073.stdout:3875 read 0x6ae2a14 thru 0x6aec350 (0x993d bytes) 2024-09-21T14:22:07.364 INFO:teuthology.orchestra.run.smithi073.stdout:3877 read 0x8c93846 thru 0x8c982d0 (0x4a8b bytes) 2024-09-21T14:22:07.365 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:07.366 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:07.375 INFO:teuthology.orchestra.run.smithi073.stdout:3878 write 0xe4ec975 thru 0xe4f5700 (0x8d8c bytes) 2024-09-21T14:22:07.378 INFO:teuthology.orchestra.run.smithi073.stdout:3880 write 0x9518d28 thru 0x952793e (0xec17 bytes) 2024-09-21T14:22:07.381 INFO:teuthology.orchestra.run.smithi073.stdout:3881 read 0xd7015ba thru 0xd70c80e (0xb255 bytes) 2024-09-21T14:22:07.381 INFO:teuthology.orchestra.run.smithi073.stdout:3882 punch from 0x28c4537 to 0x28cd27b, (0x8d44 bytes) 2024-09-21T14:22:07.381 INFO:teuthology.orchestra.run.smithi073.stdout:3884 write 0x30820d9 thru 0x308c95b (0xa883 bytes) 2024-09-21T14:22:07.383 INFO:teuthology.orchestra.run.smithi073.stdout:3885 punch from 0x325ae79 to 0x326143c, (0x65c3 bytes) 2024-09-21T14:22:07.383 INFO:teuthology.orchestra.run.smithi073.stdout:3887 write 0xb8bfca5 thru 0xb8c5a1f (0x5d7b bytes) 2024-09-21T14:22:07.383 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:07.389 INFO:teuthology.orchestra.run.smithi073.stdout:3889 write 0x549a25 thru 0x54d044 (0x3620 bytes) 2024-09-21T14:22:07.391 INFO:teuthology.orchestra.run.smithi073.stdout:3890 write 0x3ca4faf thru 0x3cada9d (0x8aef bytes) 2024-09-21T14:22:07.392 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.393 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:07.393+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:07.401 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:07.401 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:07.406 INFO:teuthology.orchestra.run.smithi073.stdout:3892 trunc from 0xe4f5701 to 0x29466f6 2024-09-21T14:22:07.416 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706584286999998, 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:22:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706572977, 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:22:07.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:22:07.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706557601, 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:22:07.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:22:07.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706540072999999, 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:22:07.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:22:07.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706521121999998, 2024-09-21T14:22:07.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:22:07.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:22:07.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706504742, 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.442 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706495513999997, 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.443 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:22:07.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:22:07.446 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706484117999999, 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.447 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706461390999998, 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.451 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706445135000003, 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.455 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:22:07.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.70643038, 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:22:07.459 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:22:07.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706421325999997, 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:07.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:22:07.467 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706410282, 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.468 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:22:07.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706394881000001, 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.472 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.476 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706378182999998, 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:07.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.480 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706362575999997, 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.481 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:22:07.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706345454000001, 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:22:07.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706326842999999, 2024-09-21T14:22:07.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.493 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706307512999999, 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:07.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.706287875999998, 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.498 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:07.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:07.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:22:07.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:07.502 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:07.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:22:07.503 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:07.503 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:07.504 INFO:teuthology.orchestra.run.smithi073.stdout:3893 write 0x2db8033 thru 0x2dc7c8e (0xfc5c bytes) 2024-09-21T14:22:07.504 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.504 INFO:teuthology.orchestra.run.smithi073.stdout:3896 read 0xf739b2 thru 0xf7c499 (0x8ae8 bytes) 2024-09-21T14:22:07.504 INFO:teuthology.orchestra.run.smithi073.stdout:3897 writesame 0x2879a9b thru 0x288404d data_size 0xd(0xa5b3 bytes) 2024-09-21T14:22:07.505 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:07.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:07.505 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:07.504+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:07.506 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:07.506 INFO:teuthology.orchestra.run.smithi073.stderr:set 1.1 pg_upmap mapping to [5,4] 2024-09-21T14:22:07.506 INFO:teuthology.orchestra.run.smithi073.stdout:3898 write 0xa06ddcf thru 0xa077fe8 (0xa21a bytes) 2024-09-21T14:22:07.506 INFO:teuthology.orchestra.run.smithi073.stdout:3900 read 0x2388960 thru 0x2389717 (0xdb8 bytes) 2024-09-21T14:22:07.507 INFO:teuthology.orchestra.run.smithi073.stdout:3901 write 0x52c1940 thru 0x52cec07 (0xd2c8 bytes) 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:3902 punch from 0x16c1d69 to 0x16c22be, (0x555 bytes) 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:3903 punch from 0x3537484 to 0x3546622, (0xf19e bytes) 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:3904 read 0x54b6678 thru 0x54b6862 (0x1eb bytes) 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:3906 read 0x208ab3 thru 0x215305 (0xc853 bytes) 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:3907 trunc from 0xa077fe9 to 0xa6a88e5 2024-09-21T14:22:07.508 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:07.509 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:07.509 INFO:teuthology.orchestra.run.smithi073.stdout:3910 write 0xc36aecb thru 0xc376ab0 (0xbbe6 bytes) 2024-09-21T14:22:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:3912 read 0x15fe2eb thru 0x160a936 (0xc64c bytes) 2024-09-21T14:22:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:3914 write 0x70be7a6 thru 0x70cb947 (0xd1a2 bytes) 2024-09-21T14:22:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:3915 write 0xe8b42dd thru 0xe8b729d (0x2fc1 bytes) 2024-09-21T14:22:07.510 INFO:teuthology.orchestra.run.smithi073.stdout:3916 trunc from 0xe8b729e to 0x57b5b76 2024-09-21T14:22:07.520 INFO:teuthology.orchestra.run.smithi073.stdout:3918 trunc from 0x57b5b76 to 0x487b063 2024-09-21T14:22:07.529 INFO:teuthology.orchestra.run.smithi073.stdout:3920 trunc from 0x487b063 to 0x22d13d8 2024-09-21T14:22:07.543 INFO:teuthology.orchestra.run.smithi073.stdout:3922 read 0x186d528 thru 0x186f164 (0x1c3d bytes) 2024-09-21T14:22:07.552 INFO:teuthology.orchestra.run.smithi073.stdout:3923 trunc from 0x22d13d8 to 0x93d4c83 2024-09-21T14:22:07.565 INFO:teuthology.orchestra.run.smithi073.stdout:3924 writesame 0xc53941b thru 0xc53dd4d data_size 0x1(0x4933 bytes) 2024-09-21T14:22:07.570 INFO:teuthology.orchestra.run.smithi073.stdout:3927 write 0xc58312a thru 0xc58c48f (0x9366 bytes) 2024-09-21T14:22:07.582 INFO:teuthology.orchestra.run.smithi073.stdout:3928 write 0xeb7834a thru 0xeb85638 (0xd2ef bytes) 2024-09-21T14:22:07.590 INFO:teuthology.orchestra.run.smithi073.stdout:3929 trunc from 0xeb85639 to 0xa8a46a1 2024-09-21T14:22:07.605 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:07.605+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:07.606 INFO:teuthology.orchestra.run.smithi073.stdout:3930 trunc from 0xa8a46a1 to 0xd8662d2 2024-09-21T14:22:07.611 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.611 INFO:teuthology.orchestra.run.smithi073.stdout:3933 read 0xb8f62bd thru 0xb903cca (0xda0e bytes) 2024-09-21T14:22:07.611 INFO:teuthology.orchestra.run.smithi073.stdout:3934 read 0x7e3afb7 thru 0x7e3e442 (0x348c bytes) 2024-09-21T14:22:07.611 INFO:teuthology.orchestra.run.smithi073.stdout:3935 writesame 0x2e50369 thru 0x2e582ac data_size 0x5a(0x7f44 bytes) 2024-09-21T14:22:07.622 INFO:teuthology.orchestra.run.smithi073.stdout:3936 trunc from 0xd8662d2 to 0x65c1d81 2024-09-21T14:22:07.632 INFO:teuthology.orchestra.run.smithi073.stdout:3937 write 0x2f92cff thru 0x2fa20b3 (0xf3b5 bytes) 2024-09-21T14:22:07.642 INFO:teuthology.orchestra.run.smithi073.stdout:3938 write 0x8e732f3 thru 0x8e79936 (0x6644 bytes) 2024-09-21T14:22:07.645 INFO:teuthology.orchestra.run.smithi073.stdout:3939 read 0x4316ad3 thru 0x431e5de (0x7b0c bytes) 2024-09-21T14:22:07.645 INFO:teuthology.orchestra.run.smithi073.stdout:3940 read 0x1ed570c thru 0x1edc05a (0x694f bytes) 2024-09-21T14:22:07.646 INFO:teuthology.orchestra.run.smithi073.stdout:3941 trunc from 0x8e79937 to 0x802116f 2024-09-21T14:22:07.651 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:07.651 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:07.653 INFO:teuthology.orchestra.run.smithi073.stdout:3943 write 0x11684f5 thru 0x117748e (0xef9a bytes) 2024-09-21T14:22:07.655 INFO:teuthology.orchestra.run.smithi073.stdout:3946 punch from 0x5768d53 to 0x577453e, (0xb7eb bytes) 2024-09-21T14:22:07.655 INFO:teuthology.orchestra.run.smithi073.stdout:3948 write 0x75a90ab thru 0x75af689 (0x65df bytes) 2024-09-21T14:22:07.658 INFO:teuthology.orchestra.run.smithi073.stdout:3949 read 0x76a5ddd thru 0x76a6db3 (0xfd7 bytes) 2024-09-21T14:22:07.660 INFO:teuthology.orchestra.run.smithi073.stdout:3950 trunc from 0x802116f to 0x6ca3191 2024-09-21T14:22:07.661 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:07.670 INFO:teuthology.orchestra.run.smithi073.stdout:3951 read 0x5c9fc1d thru 0x5cab04d (0xb431 bytes) 2024-09-21T14:22:07.670 INFO:teuthology.orchestra.run.smithi073.stdout:3953 write 0x570ddde thru 0x5719c37 (0xbe5a bytes) 2024-09-21T14:22:07.680 INFO:teuthology.orchestra.run.smithi073.stdout:3955 trunc from 0x6ca3191 to 0xd22de56 2024-09-21T14:22:07.686 INFO:teuthology.orchestra.run.smithi073.stdout:3956 write 0xcc8e417 thru 0xcc956fd (0x72e7 bytes) 2024-09-21T14:22:07.689 INFO:teuthology.orchestra.run.smithi073.stdout:3957 writesame 0xd1dffe2 thru 0xd1ebd5a data_size 0x59(0xbd79 bytes) 2024-09-21T14:22:07.692 INFO:teuthology.orchestra.run.smithi073.stdout:3960 read 0x97fc4ee thru 0x9801b35 (0x5648 bytes) 2024-09-21T14:22:07.692 INFO:teuthology.orchestra.run.smithi073.stdout:3962 writesame 0x3d6fd9e thru 0x3d7aeb9 data_size 0xa(0xb11c bytes) 2024-09-21T14:22:07.696 INFO:teuthology.orchestra.run.smithi073.stdout:3963 read 0xb514f98 thru 0xb52141c (0xc485 bytes) 2024-09-21T14:22:07.703 INFO:teuthology.orchestra.run.smithi073.stdout:3965 trunc from 0xd22de56 to 0x71867f4 2024-09-21T14:22:07.706 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:07.706+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:07.716 INFO:teuthology.orchestra.run.smithi073.stdout:3968 trunc from 0x71867f4 to 0xa0a9c2d 2024-09-21T14:22:07.723 INFO:teuthology.orchestra.run.smithi073.stdout:3969 read 0x272ae25 thru 0x272df5d (0x3139 bytes) 2024-09-21T14:22:07.723 INFO:teuthology.orchestra.run.smithi073.stdout:3970 read 0x10665c8 thru 0x106bccb (0x5704 bytes) 2024-09-21T14:22:07.724 INFO:teuthology.orchestra.run.smithi073.stdout:3971 read 0x4b98e6a thru 0x4b9dea8 (0x503f bytes) 2024-09-21T14:22:07.724 INFO:teuthology.orchestra.run.smithi073.stdout:3974 trunc from 0xa0a9c2d to 0x9e20f82 2024-09-21T14:22:07.729 INFO:teuthology.orchestra.run.smithi073.stdout:3975 read 0xfa1041 thru 0xfa6169 (0x5129 bytes) 2024-09-21T14:22:07.738 INFO:teuthology.orchestra.run.smithi073.stdout:3977 trunc from 0x9e20f82 to 0x2626682 2024-09-21T14:22:07.765 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:07.765 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:07.770 INFO:teuthology.orchestra.run.smithi073.stdout:3978 write 0x90a3f76 thru 0x90af6fc (0xb787 bytes) 2024-09-21T14:22:07.773 INFO:teuthology.orchestra.run.smithi073.stdout:3982 writesame 0x4a956e2 thru 0x4a9c7a4 data_size 0x1(0x70c3 bytes) 2024-09-21T14:22:07.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:07.794 INFO:teuthology.orchestra.run.smithi073.stdout:3983 trunc from 0x90af6fd to 0xe2fe5ca 2024-09-21T14:22:07.799 INFO:teuthology.orchestra.run.smithi073.stdout:3985 read 0x43b843a thru 0x43bccbe (0x4885 bytes) 2024-09-21T14:22:07.799 INFO:teuthology.orchestra.run.smithi073.stdout:3986 write 0xe257ecd thru 0xe258242 (0x376 bytes) 2024-09-21T14:22:07.802 INFO:teuthology.orchestra.run.smithi073.stdout:3988 write 0x37274a7 thru 0x3736111 (0xec6b bytes) 2024-09-21T14:22:07.804 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.804 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.811 INFO:teuthology.orchestra.run.smithi073.stdout:3992 trunc from 0xe2fe5ca to 0x87fb740 2024-09-21T14:22:07.815 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:07.815+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:07.822 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.822 INFO:teuthology.orchestra.run.smithi073.stdout:3994 read 0x47751fc thru 0x4776b14 (0x1919 bytes) 2024-09-21T14:22:07.822 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.822 INFO:teuthology.orchestra.run.smithi073.stdout:3996 write 0x62b25fc thru 0x62b8d15 (0x671a bytes) 2024-09-21T14:22:07.826 INFO:teuthology.orchestra.run.smithi073.stdout:3997 punch from 0x2e756b9 to 0x2e76f49, (0x1890 bytes) 2024-09-21T14:22:07.832 INFO:teuthology.orchestra.run.smithi073.stdout:3998 write 0x954fd1f thru 0x9557b00 (0x7de2 bytes) 2024-09-21T14:22:07.835 INFO:teuthology.orchestra.run.smithi073.stdout:3999 write 0x3cb055c thru 0x3cb6bc8 (0x666d bytes) 2024-09-21T14:22:07.838 INFO:teuthology.orchestra.run.smithi073.stdout:4001 write 0x792f738 thru 0x7932187 (0x2a50 bytes) 2024-09-21T14:22:07.840 INFO:teuthology.orchestra.run.smithi073.stdout:4002 read 0x966cdf thru 0x97064b (0x996d bytes) 2024-09-21T14:22:07.841 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.841 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.841 INFO:teuthology.orchestra.run.smithi073.stdout:4005 write 0x2db80bc thru 0x2dc5564 (0xd4a9 bytes) 2024-09-21T14:22:07.845 INFO:teuthology.orchestra.run.smithi073.stdout:4006 writesame 0x91d9dc4 thru 0x91e6094 data_size 0x1(0xc2d1 bytes) 2024-09-21T14:22:07.862 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.871 INFO:teuthology.orchestra.run.smithi073.stdout:4009 write 0xbd69aa0 thru 0xbd71924 (0x7e85 bytes) 2024-09-21T14:22:07.874 INFO:teuthology.orchestra.run.smithi073.stdout:4010 read 0x1baf15 thru 0x1c778d (0xc879 bytes) 2024-09-21T14:22:07.874 INFO:teuthology.orchestra.run.smithi073.stdout:4011 writesame 0x90c90ef thru 0x90d869d data_size 0x23(0xf5af bytes) 2024-09-21T14:22:07.876 INFO:teuthology.orchestra.run.smithi073.stdout:4012 write 0x93ba895 thru 0x93c0c73 (0x63df bytes) 2024-09-21T14:22:07.888 INFO:teuthology.orchestra.run.smithi073.stdout:4013 write 0xe9cbec5 thru 0xe9cec6e (0x2daa bytes) 2024-09-21T14:22:07.891 INFO:teuthology.orchestra.run.smithi073.stdout:4014 write 0xb59a95a thru 0xb5a4074 (0x971b bytes) 2024-09-21T14:22:07.894 INFO:teuthology.orchestra.run.smithi073.stdout:4015 write 0x444cc24 thru 0x445b2ce (0xe6ab bytes) 2024-09-21T14:22:07.897 INFO:teuthology.orchestra.run.smithi073.stdout:4016 write 0x1cd1621 thru 0x1cda80c (0x91ec bytes) 2024-09-21T14:22:07.899 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.899 INFO:teuthology.orchestra.run.smithi073.stdout:4018 writesame 0xa9cfa55 thru 0xa9d1369 data_size 0x1(0x1915 bytes) 2024-09-21T14:22:07.904 INFO:teuthology.orchestra.run.smithi073.stdout:4019 write 0xd48c97c thru 0xd48d03d (0x6c2 bytes) 2024-09-21T14:22:07.906 INFO:teuthology.orchestra.run.smithi073.stdout:4020 write 0xc67698a thru 0xc67f927 (0x8f9e bytes) 2024-09-21T14:22:07.907 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:07.907+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:07.909 INFO:teuthology.orchestra.run.smithi073.stdout:4023 read 0x28adfa2 thru 0x28bc8ee (0xe94d bytes) 2024-09-21T14:22:07.909 INFO:teuthology.orchestra.run.smithi073.stdout:4024 write 0x11feaa5 thru 0x1208802 (0x9d5e bytes) 2024-09-21T14:22:07.911 INFO:teuthology.orchestra.run.smithi073.stdout:4025 writesame 0x3c97f80 thru 0x3ca389d data_size 0x23(0xb91e bytes) 2024-09-21T14:22:07.913 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.913 INFO:teuthology.orchestra.run.smithi073.stdout:4028 read 0xa2031f3 thru 0xa209fd9 (0x6de7 bytes) 2024-09-21T14:22:07.913 INFO:teuthology.orchestra.run.smithi073.stdout:4029 write 0xd941f72 thru 0xd94e442 (0xc4d1 bytes) 2024-09-21T14:22:07.922 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:07.922 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:07.929 INFO:teuthology.orchestra.run.smithi073.stdout:4032 trunc from 0xe9cec6f to 0x2dc5f29 2024-09-21T14:22:07.931 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:07.944 INFO:teuthology.orchestra.run.smithi073.stdout:4033 writesame 0xe5e2d2 thru 0xe6125d data_size 0x11(0x2f8c bytes) 2024-09-21T14:22:07.945 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:07.963 INFO:teuthology.orchestra.run.smithi073.stdout:4037 writesame 0xd7e0bdb thru 0xd7e7368 data_size 0x37(0x678e bytes) 2024-09-21T14:22:07.963 INFO:teuthology.orchestra.run.smithi073.stdout:4038 write 0x98fe371 thru 0x990af18 (0xcba8 bytes) 2024-09-21T14:22:07.967 INFO:teuthology.orchestra.run.smithi073.stdout:4039 read 0x286bda0 thru 0x2870dd0 (0x5031 bytes) 2024-09-21T14:22:07.976 INFO:teuthology.orchestra.run.smithi073.stdout:4040 trunc from 0xd7e7369 to 0x698cc14 2024-09-21T14:22:07.987 INFO:teuthology.orchestra.run.smithi073.stdout:4041 punch from 0x1283b50 to 0x1292f58, (0xf408 bytes) 2024-09-21T14:22:07.987 INFO:teuthology.orchestra.run.smithi073.stdout:4042 punch from 0x3eaf32d to 0x3ebaba5, (0xb878 bytes) 2024-09-21T14:22:07.987 INFO:teuthology.orchestra.run.smithi073.stdout:4043 punch from 0xadce33 to 0xade8e7, (0x1ab4 bytes) 2024-09-21T14:22:07.987 INFO:teuthology.orchestra.run.smithi073.stdout:4044 read 0x3c7d563 thru 0x3c88e41 (0xb8df bytes) 2024-09-21T14:22:07.997 INFO:teuthology.orchestra.run.smithi073.stdout:4045 writesame 0xa12ca70 thru 0xa130ea6 data_size 0x1(0x4437 bytes) 2024-09-21T14:22:07.997 INFO:teuthology.orchestra.run.smithi073.stdout:4048 writesame 0x58a827a thru 0x58a83df data_size 0x1(0x166 bytes) 2024-09-21T14:22:08.007 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:08.008+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:08.008 INFO:teuthology.orchestra.run.smithi073.stdout:4049 write 0xc965982 thru 0xc96d670 (0x7cef bytes) 2024-09-21T14:22:08.011 INFO:teuthology.orchestra.run.smithi073.stdout:4052 read 0xae890ed thru 0xae8c1d4 (0x30e8 bytes) 2024-09-21T14:22:08.011 INFO:teuthology.orchestra.run.smithi073.stdout:4053 write 0x54c224 thru 0x55213a (0x5f17 bytes) 2024-09-21T14:22:08.020 INFO:teuthology.orchestra.run.smithi073.stdout:4054 writesame 0xceb6f4b thru 0xcebe86d data_size 0x1(0x7923 bytes) 2024-09-21T14:22:08.021 INFO:teuthology.orchestra.run.smithi073.stdout:4055 read 0xc645800 thru 0xc64d3c9 (0x7bca bytes) 2024-09-21T14:22:08.022 INFO:teuthology.orchestra.run.smithi073.stdout:4057 read 0x1f026ed thru 0x1f0d3a2 (0xacb6 bytes) 2024-09-21T14:22:08.022 INFO:teuthology.orchestra.run.smithi073.stdout:4059 read 0x6393947 thru 0x639deef (0xa5a9 bytes) 2024-09-21T14:22:08.033 INFO:teuthology.orchestra.run.smithi073.stdout:4061 trunc from 0xcebe86e to 0x386ebf6 2024-09-21T14:22:08.038 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.038 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.043 INFO:teuthology.orchestra.run.smithi073.stdout:4063 read 0x26aff94 thru 0x26b49ee (0x4a5b bytes) 2024-09-21T14:22:08.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:08.048 INFO:teuthology.orchestra.run.smithi073.stdout:4064 trunc from 0x386ebf6 to 0x7286902 2024-09-21T14:22:08.053 INFO:teuthology.orchestra.run.smithi073.stdout:4065 read 0x21ab41 thru 0x225bd7 (0xb097 bytes) 2024-09-21T14:22:08.054 INFO:teuthology.orchestra.run.smithi073.stdout:4066 writesame 0x336974a thru 0x337289d data_size 0x4(0x9154 bytes) 2024-09-21T14:22:08.059 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.067 INFO:teuthology.orchestra.run.smithi073.stdout:4069 write 0x98017ca thru 0x980fcff (0xe536 bytes) 2024-09-21T14:22:08.070 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.070 INFO:teuthology.orchestra.run.smithi073.stdout:4072 read 0x789358f thru 0x789942e (0x5ea0 bytes) 2024-09-21T14:22:08.070 INFO:teuthology.orchestra.run.smithi073.stdout:4073 write 0x2e375d2 thru 0x2e472ed (0xfd1c bytes) 2024-09-21T14:22:08.074 INFO:teuthology.orchestra.run.smithi073.stdout:4074 writesame 0x355b3a0 thru 0x356a3ac data_size 0x1(0xf00d bytes) 2024-09-21T14:22:08.094 INFO:teuthology.orchestra.run.smithi073.stdout:4075 clone 34 order 19 su 16384 sc 14 2024-09-21T14:22:08.109 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:08.109+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:08.188 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.189 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:08.209 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:08.210+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:08.298 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.299 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.307 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:08.310 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:08.310+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:08.410 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:08.410+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:08.411 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.411 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:08.445 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:22:08.511 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:08.511+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:08.519 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone33 from 0x980fd00 (overlap 0xc95b64) to 0x8cf32c 2024-09-21T14:22:08.571 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.571 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:08.609 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T14:22:08.617 INFO:teuthology.orchestra.run.smithi073.stdout:4076 read 0x3a67723 thru 0x3a7331b (0xbbf9 bytes) 2024-09-21T14:22:08.618 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:08.618+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:08.627 INFO:teuthology.orchestra.run.smithi073.stdout:4077 write 0xad5fac2 thru 0xad606f3 (0xc32 bytes) 2024-09-21T14:22:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:4078 read 0x1d02922 thru 0x1d0560a (0x2ce9 bytes) 2024-09-21T14:22:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:4079 write 0x4e87230 thru 0x4e8a16e (0x2f3f bytes) 2024-09-21T14:22:08.645 INFO:teuthology.orchestra.run.smithi073.stdout:4080 writesame 0xb23e13d thru 0xb24366e data_size 0xf(0x5532 bytes) 2024-09-21T14:22:08.645 INFO:teuthology.orchestra.run.smithi073.stdout:4081 punch from 0x7416efd to 0x7418447, (0x154a bytes) 2024-09-21T14:22:08.645 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.645 INFO:teuthology.orchestra.run.smithi073.stdout:4084 write 0x3cd3948 thru 0x3ce2f14 (0xf5cd bytes) 2024-09-21T14:22:08.654 INFO:teuthology.orchestra.run.smithi073.stdout:4085 trunc from 0xb24366f to 0xc7a575b 2024-09-21T14:22:08.661 INFO:teuthology.orchestra.run.smithi073.stdout:4086 trunc from 0xc7a575b to 0xcf250e8 2024-09-21T14:22:08.667 INFO:teuthology.orchestra.run.smithi073.stdout:4087 read 0x66855c0 thru 0x668cd13 (0x7754 bytes) 2024-09-21T14:22:08.667 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.667 INFO:teuthology.orchestra.run.smithi073.stdout:4089 read 0x308eb63 thru 0x3092c71 (0x410f bytes) 2024-09-21T14:22:08.667 INFO:teuthology.orchestra.run.smithi073.stdout:4090 read 0x12ce781 thru 0x12db7bc (0xd03c bytes) 2024-09-21T14:22:08.668 INFO:teuthology.orchestra.run.smithi073.stdout:4095 read 0x3883642 thru 0x389020b (0xcbca bytes) 2024-09-21T14:22:08.668 INFO:teuthology.orchestra.run.smithi073.stdout:4096 write 0x3922651 thru 0x3930c40 (0xe5f0 bytes) 2024-09-21T14:22:08.674 INFO:teuthology.orchestra.run.smithi073.stdout:4097 read 0x457c11e thru 0x45895ba (0xd49d bytes) 2024-09-21T14:22:08.681 INFO:teuthology.orchestra.run.smithi073.stdout:4098 trunc from 0xcf250e8 to 0x76ef73f 2024-09-21T14:22:08.697 INFO:teuthology.orchestra.run.smithi073.stdout:4099 read 0x16ea701 thru 0x16ed0a6 (0x29a6 bytes) 2024-09-21T14:22:08.697 INFO:teuthology.orchestra.run.smithi073.stdout:4100 read 0x50043f4 thru 0x500f0c6 (0xacd3 bytes) 2024-09-21T14:22:08.698 INFO:teuthology.orchestra.run.smithi073.stdout:4101 read 0x3b37805 thru 0x3b3c536 (0x4d32 bytes) 2024-09-21T14:22:08.713 INFO:teuthology.orchestra.run.smithi073.stdout:4102 write 0xee2e517 thru 0xee30663 (0x214d bytes) 2024-09-21T14:22:08.714 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:08.714+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:08.715 INFO:teuthology.orchestra.run.smithi073.stdout:4106 read 0x3247745 thru 0x32508df (0x919b bytes) 2024-09-21T14:22:08.716 INFO:teuthology.orchestra.run.smithi073.stdout:4107 writesame 0xe0d1c1 thru 0xe102f0 data_size 0x8(0x3130 bytes) 2024-09-21T14:22:08.724 INFO:teuthology.orchestra.run.smithi073.stdout:4108 punch from 0x1819bd0 to 0x181b201, (0x1631 bytes) 2024-09-21T14:22:08.727 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.727 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.732 INFO:teuthology.orchestra.run.smithi073.stdout:4109 trunc from 0xee30664 to 0x84c55a0 2024-09-21T14:22:08.739 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:08.741 INFO:teuthology.orchestra.run.smithi073.stdout:4110 punch from 0x7c0265a to 0x7c1038c, (0xdd32 bytes) 2024-09-21T14:22:08.741 INFO:teuthology.orchestra.run.smithi073.stdout:4112 punch from 0x81a0548 to 0x81a2a06, (0x24be bytes) 2024-09-21T14:22:08.741 INFO:teuthology.orchestra.run.smithi073.stdout:4113 write 0x63d3aeb thru 0x63e01d7 (0xc6ed bytes) 2024-09-21T14:22:08.750 INFO:teuthology.orchestra.run.smithi073.stdout:4114 write 0x5b4d281 thru 0x5b557ca (0x854a bytes) 2024-09-21T14:22:08.767 INFO:teuthology.orchestra.run.smithi073.stdout:4116 write 0xb5b75e7 thru 0xb5bf698 (0x80b2 bytes) 2024-09-21T14:22:08.775 INFO:teuthology.orchestra.run.smithi073.stdout:4118 read 0xa239cd3 thru 0xa243c9a (0x9fc8 bytes) 2024-09-21T14:22:08.783 INFO:teuthology.orchestra.run.smithi073.stdout:4119 writesame 0xcd16006 thru 0xcd21d17 data_size 0x9(0xbd12 bytes) 2024-09-21T14:22:08.785 INFO:teuthology.orchestra.run.smithi073.stdout:4120 trunc from 0xcd21d18 to 0xe50e0ab 2024-09-21T14:22:08.790 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.790 INFO:teuthology.orchestra.run.smithi073.stdout:4124 write 0x70ca555 thru 0x70d3fd6 (0x9a82 bytes) 2024-09-21T14:22:08.794 INFO:teuthology.orchestra.run.smithi073.stdout:4125 read 0x6ff097e thru 0x6ff9fd2 (0x9655 bytes) 2024-09-21T14:22:08.795 INFO:teuthology.orchestra.run.smithi073.stdout:4126 read 0xd42837d thru 0xd429efc (0x1b80 bytes) 2024-09-21T14:22:08.795 INFO:teuthology.orchestra.run.smithi073.stdout:4127 writesame 0x4bd47df thru 0x4be050e data_size 0x18(0xbd30 bytes) 2024-09-21T14:22:08.815 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:08.815+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:08.817 INFO:teuthology.orchestra.run.smithi073.stdout:4128 trunc from 0xe50e0ab to 0x6bf4127 2024-09-21T14:22:08.850 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.850 INFO:teuthology.orchestra.run.smithi073.stdout:4133 punch from 0x5017cf6 to 0x5022319, (0xa623 bytes) 2024-09-21T14:22:08.850 INFO:teuthology.orchestra.run.smithi073.stdout:4134 read 0xa5dab7 thru 0xa61a37 (0x3f81 bytes) 2024-09-21T14:22:08.850 INFO:teuthology.orchestra.run.smithi073.stdout:4135 read 0x5566f1e thru 0x5569d54 (0x2e37 bytes) 2024-09-21T14:22:08.850 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:08.851 INFO:teuthology.orchestra.run.smithi073.stdout:4139 punch from 0x7566c4 to 0x75944b, (0x2d87 bytes) 2024-09-21T14:22:08.851 INFO:teuthology.orchestra.run.smithi073.stdout:4142 writesame 0xc7049b thru 0xc7fc2a data_size 0x88(0xf790 bytes) 2024-09-21T14:22:08.863 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:08.863 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:08.864 INFO:teuthology.orchestra.run.smithi073.stdout:4143 writesame 0x68bef23 thru 0x68ca8ff data_size 0x1(0xb9dd bytes) 2024-09-21T14:22:08.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:08.891 INFO:teuthology.orchestra.run.smithi073.stdout:4144 write 0x959c8de thru 0x95a7cd3 (0xb3f6 bytes) 2024-09-21T14:22:08.909 INFO:teuthology.orchestra.run.smithi073.stdout:4145 writesame 0xe617381 thru 0xe61c8c2 data_size 0x7(0x5542 bytes) 2024-09-21T14:22:08.909 INFO:teuthology.orchestra.run.smithi073.stdout:4146 read 0xd8a9fa4 thru 0xd8aa434 (0x491 bytes) 2024-09-21T14:22:08.909 INFO:teuthology.orchestra.run.smithi073.stdout:4148 writesame 0x42b3feb thru 0x42be9c8 data_size 0x11(0xa9de bytes) 2024-09-21T14:22:08.916 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:08.916+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:08.918 INFO:teuthology.orchestra.run.smithi073.stdout:4149 punch from 0xdc70295 to 0xdc7949f, (0x920a bytes) 2024-09-21T14:22:08.918 INFO:teuthology.orchestra.run.smithi073.stdout:4150 punch from 0x360881a to 0x360dad1, (0x52b7 bytes) 2024-09-21T14:22:08.918 INFO:teuthology.orchestra.run.smithi073.stdout:4151 read 0x7230a1b thru 0x7233ad4 (0x30ba bytes) 2024-09-21T14:22:08.918 INFO:teuthology.orchestra.run.smithi073.stdout:4153 read 0x9ccbf4f thru 0x9cdad24 (0xedd6 bytes) 2024-09-21T14:22:08.918 INFO:teuthology.orchestra.run.smithi073.stdout:4154 punch from 0x5b1ac22 to 0x5b284c2, (0xd8a0 bytes) 2024-09-21T14:22:08.919 INFO:teuthology.orchestra.run.smithi073.stdout:4155 write 0x39750b4 thru 0x3980fad (0xbefa bytes) 2024-09-21T14:22:08.929 INFO:teuthology.orchestra.run.smithi073.stdout:4156 write 0x4134a47 thru 0x413d5c9 (0x8b83 bytes) 2024-09-21T14:22:08.935 INFO:teuthology.orchestra.run.smithi073.stdout:4157 writesame 0x91a84ed thru 0x91b0691 data_size 0x27(0x81a5 bytes) 2024-09-21T14:22:08.941 INFO:teuthology.orchestra.run.smithi073.stdout:4158 writesame 0x1d9950c thru 0x1d9bfcf data_size 0x2e(0x2ac4 bytes) 2024-09-21T14:22:08.946 INFO:teuthology.orchestra.run.smithi073.stdout:4161 write 0x57c28a2 thru 0x57c98ac (0x700b bytes) 2024-09-21T14:22:08.952 INFO:teuthology.orchestra.run.smithi073.stdout:4162 read 0x3d7c357 thru 0x3d83ebd (0x7b67 bytes) 2024-09-21T14:22:08.959 INFO:teuthology.orchestra.run.smithi073.stdout:4163 trunc from 0xe61c8c3 to 0x8ca8bf5 2024-09-21T14:22:08.970 INFO:teuthology.orchestra.run.smithi073.stdout:4164 read 0x5a1734c thru 0x5a1e5ec (0x72a1 bytes) 2024-09-21T14:22:08.970 INFO:teuthology.orchestra.run.smithi073.stdout:4165 read 0x7318533 thru 0x7327ce4 (0xf7b2 bytes) 2024-09-21T14:22:08.970 INFO:teuthology.orchestra.run.smithi073.stdout:4166 read 0x48dd6f1 thru 0x48e51e7 (0x7af7 bytes) 2024-09-21T14:22:08.970 INFO:teuthology.orchestra.run.smithi073.stdout:4167 writesame 0x2ccf18f thru 0x2cd38fb data_size 0x45(0x476d bytes) 2024-09-21T14:22:08.981 INFO:teuthology.orchestra.run.smithi073.stdout:4168 trunc from 0x8ca8bf5 to 0x5990068 2024-09-21T14:22:09.013 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:09.013 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:09.016 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:09.016+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:09.023 INFO:teuthology.orchestra.run.smithi073.stdout:4169 trunc from 0x5990068 to 0xe960a0c 2024-09-21T14:22:09.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:09.031 INFO:teuthology.orchestra.run.smithi073.stdout:4172 read 0x4dbd4d7 thru 0x4dc3874 (0x639e bytes) 2024-09-21T14:22:09.031 INFO:teuthology.orchestra.run.smithi073.stdout:4173 write 0x7d05fb4 thru 0x7d0fc92 (0x9cdf bytes) 2024-09-21T14:22:09.045 INFO:teuthology.orchestra.run.smithi073.stdout:4174 trunc from 0xe960a0c to 0x7cac2e9 2024-09-21T14:22:09.051 INFO:teuthology.orchestra.run.smithi073.stdout:4175 read 0x6edaf84 thru 0x6ee90df (0xe15c bytes) 2024-09-21T14:22:09.051 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.052 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.064 INFO:teuthology.orchestra.run.smithi073.stdout:4178 write 0xce41766 thru 0xce4428a (0x2b25 bytes) 2024-09-21T14:22:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:4179 writesame 0x7c9161f thru 0x7c9c0c6 data_size 0x56(0xaaa8 bytes) 2024-09-21T14:22:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:4181 read 0x60dd31d thru 0x60ddaf2 (0x7d6 bytes) 2024-09-21T14:22:09.089 INFO:teuthology.orchestra.run.smithi073.stdout:4182 trunc from 0xce4428b to 0xaea373 2024-09-21T14:22:09.117 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:09.117+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:09.153 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:09.153 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:09.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:09.246 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:09.246+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:09.304 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:09.304 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:09.313 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:09.345 INFO:teuthology.orchestra.run.smithi073.stdout:4183 write 0x62af5fc thru 0x62bc336 (0xcd3b bytes) 2024-09-21T14:22:09.346 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:09.346+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:09.351 INFO:teuthology.orchestra.run.smithi073.stdout:4185 read 0x5915d16 thru 0x59224cf (0xc7ba bytes) 2024-09-21T14:22:09.351 INFO:teuthology.orchestra.run.smithi073.stdout:4186 punch from 0x25ecd07 to 0x25f440c, (0x7705 bytes) 2024-09-21T14:22:09.361 INFO:teuthology.orchestra.run.smithi073.stdout:4187 trunc from 0x62bc337 to 0xeb90110 2024-09-21T14:22:09.366 INFO:teuthology.orchestra.run.smithi073.stdout:4188 writesame 0x7074259 thru 0x7075dae data_size 0x1(0x1b56 bytes) 2024-09-21T14:22:09.371 INFO:teuthology.orchestra.run.smithi073.stdout:4189 read 0x426429a thru 0x42672ba (0x3021 bytes) 2024-09-21T14:22:09.371 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.371 INFO:teuthology.orchestra.run.smithi073.stdout:4191 write 0xaa31c1a thru 0xaa3702c (0x5413 bytes) 2024-09-21T14:22:09.374 INFO:teuthology.orchestra.run.smithi073.stdout:4192 writesame 0xce2feba thru 0xce37e72 data_size 0x3f(0x7fb9 bytes) 2024-09-21T14:22:09.382 INFO:teuthology.orchestra.run.smithi073.stdout:4193 write 0x2b005ac thru 0x2b0255c (0x1fb1 bytes) 2024-09-21T14:22:09.385 INFO:teuthology.orchestra.run.smithi073.stdout:4194 write 0x9086873 thru 0x90945f8 (0xdd86 bytes) 2024-09-21T14:22:09.397 INFO:teuthology.orchestra.run.smithi073.stdout:4195 trunc from 0xeb90110 to 0xc30adab 2024-09-21T14:22:09.406 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.407 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.407 INFO:teuthology.orchestra.run.smithi073.stdout:4199 clone 35 order 21 su 65536 sc 7 2024-09-21T14:22:09.420 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:09.420+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:09.458 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:09.458 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:09.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:09.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:09.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:09.521+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:09.535 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone34 from 0xc30adab (overlap 0xaea373) to 0x8bb5f5 2024-09-21T14:22:09.621 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:09.622+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:09.630 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:09.630 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:09.639 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:09.641 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:09.642 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:09.658 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T14:22:09.662 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:09.662 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:09.675 INFO:teuthology.orchestra.run.smithi073.stdout:4200 read 0x3639a74 thru 0x363d0f8 (0x3685 bytes) 2024-09-21T14:22:09.680 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:09.688 INFO:teuthology.orchestra.run.smithi073.stdout:4202 write 0xee448bf thru 0xee4755f (0x2ca1 bytes) 2024-09-21T14:22:09.691 INFO:teuthology.orchestra.run.smithi073.stdout:4203 read 0x6e2c15a thru 0x6e2c9b7 (0x85e bytes) 2024-09-21T14:22:09.691 INFO:teuthology.orchestra.run.smithi073.stdout:4204 write 0x6e86f19 thru 0x6e96679 (0xf761 bytes) 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:4205 read 0x201be57 thru 0x201e26d (0x2417 bytes) 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:4206 punch from 0x59d7c82 to 0x59e5224, (0xd5a2 bytes) 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:4207 read 0xcda37b0 thru 0xcdb2408 (0xec59 bytes) 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:4208 read 0xba4aece thru 0xba54fd9 (0xa10c bytes) 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:4210 punch from 0x4b6dd05 to 0x4b75b50, (0x7e4b bytes) 2024-09-21T14:22:09.699 INFO:teuthology.orchestra.run.smithi073.stdout:4211 trunc from 0xee47560 to 0xdc80ac6 2024-09-21T14:22:09.699 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:09.699 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:09.709 INFO:teuthology.orchestra.run.smithi073.stdout:4212 write 0xc6b9cb2 thru 0xc6c46f0 (0xaa3f bytes) 2024-09-21T14:22:09.716 INFO:teuthology.orchestra.run.smithi073.stdout:4213 writesame 0xb95687d thru 0xb960418 data_size 0x2e(0x9b9c bytes) 2024-09-21T14:22:09.718 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:09.719 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:09.722 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:09.723+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:09.723 INFO:teuthology.orchestra.run.smithi073.stdout:4214 read 0xce71896 thru 0xce75d73 (0x44de bytes) 2024-09-21T14:22:09.723 INFO:teuthology.orchestra.run.smithi073.stdout:4215 write 0x55c2429 thru 0x55d1a64 (0xf63c bytes) 2024-09-21T14:22:09.729 INFO:teuthology.orchestra.run.smithi073.stdout:4217 write 0x68f2d98 thru 0x68ffed1 (0xd13a bytes) 2024-09-21T14:22:09.732 INFO:teuthology.orchestra.run.smithi073.stdout:4218 write 0x6b1e4d0 thru 0x6b1ea75 (0x5a6 bytes) 2024-09-21T14:22:09.734 INFO:teuthology.orchestra.run.smithi073.stdout:4220 write 0x4638ac7 thru 0x4646aaf (0xdfe9 bytes) 2024-09-21T14:22:09.735 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:09.735 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:09.743 INFO:teuthology.orchestra.run.smithi073.stdout:4221 trunc from 0xdc80ac6 to 0xd450667 2024-09-21T14:22:09.748 INFO:teuthology.orchestra.run.smithi073.stdout:4222 read 0x5994c9e thru 0x59a232d (0xd690 bytes) 2024-09-21T14:22:09.748 INFO:teuthology.orchestra.run.smithi073.stdout:4226 write 0x155b469 thru 0x1564fc2 (0x9b5a bytes) 2024-09-21T14:22:09.754 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:09.754 INFO:teuthology.orchestra.run.smithi073.stdout:4227 read 0x6c13976 thru 0x6c20329 (0xc9b4 bytes) 2024-09-21T14:22:09.755 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.755 INFO:teuthology.orchestra.run.smithi073.stdout:4230 read 0x9a2814c thru 0x9a2ceed (0x4da2 bytes) 2024-09-21T14:22:09.755 INFO:teuthology.orchestra.run.smithi073.stdout:4232 writesame 0x1c8bb95 thru 0x1c924c5 data_size 0x1(0x6931 bytes) 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:70038 3.d e67/33 3:b4d31215:::rbd_data.11216b7a362a.0000000000000010:head v 67'210, mlcod=67'208)", 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:09.760781+0000", 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.00029247499999999999, 2024-09-21T14:22:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031790399999999998, 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760781+0000", 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760781+0000", 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760781+0000", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760808+0000", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7501999999999999e-05 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760825+0000", 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6582000000000001e-05 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760833+0000", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7369999999999995e-06 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760878+0000", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4267999999999997e-05 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:09.760899+0000", 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1299999999999999e-05 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:22:09.764 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:09.766 INFO:teuthology.orchestra.run.smithi073.stdout:4235 write 0x84d1c69 thru 0x84d6a9e (0x4e36 bytes) 2024-09-21T14:22:09.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:09.770 INFO:teuthology.orchestra.run.smithi073.stdout:4237 write 0x36345d8 thru 0x3641c04 (0xd62d bytes) 2024-09-21T14:22:09.772 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:09.772 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:09.778 INFO:teuthology.orchestra.run.smithi073.stdout:4238 write 0xd3c4a4 thru 0xd42d50 (0x68ad bytes) 2024-09-21T14:22:09.797 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.809 INFO:teuthology.orchestra.run.smithi073.stdout:4240 trunc from 0xd450667 to 0x2f850ce 2024-09-21T14:22:09.823 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:09.823+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:09.850 INFO:teuthology.orchestra.run.smithi073.stdout:4241 writesame 0xccaf803 thru 0xccb5512 data_size 0x8(0x5d10 bytes) 2024-09-21T14:22:09.850 INFO:teuthology.orchestra.run.smithi073.stdout:4242 read 0xc0ef8d5 thru 0xc0fb2e0 (0xba0c bytes) 2024-09-21T14:22:09.851 INFO:teuthology.orchestra.run.smithi073.stdout:4243 punch from 0x182cd6b to 0x183182d, (0x4ac2 bytes) 2024-09-21T14:22:09.851 INFO:teuthology.orchestra.run.smithi073.stdout:4244 write 0x86fc643 thru 0x87029fd (0x63bb bytes) 2024-09-21T14:22:09.855 INFO:teuthology.orchestra.run.smithi073.stdout:4245 read 0x7511814 thru 0x751734d (0x5b3a bytes) 2024-09-21T14:22:09.855 INFO:teuthology.orchestra.run.smithi073.stdout:4246 write 0x695a06f thru 0x695eb9e (0x4b30 bytes) 2024-09-21T14:22:09.857 INFO:teuthology.orchestra.run.smithi073.stdout:4247 punch from 0x55a40ce to 0x55a7852, (0x3784 bytes) 2024-09-21T14:22:09.857 INFO:teuthology.orchestra.run.smithi073.stdout:4248 read 0xb1c5e thru 0xb8fff (0x73a2 bytes) 2024-09-21T14:22:09.858 INFO:teuthology.orchestra.run.smithi073.stdout:4250 punch from 0x1599726 to 0x159e85a, (0x5134 bytes) 2024-09-21T14:22:09.858 INFO:teuthology.orchestra.run.smithi073.stdout:4251 write 0x6c9d771 thru 0x6ca81fa (0xaa8a bytes) 2024-09-21T14:22:09.861 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.861 INFO:teuthology.orchestra.run.smithi073.stdout:4253 write 0x3ecfa7 thru 0x3ef850 (0x28aa bytes) 2024-09-21T14:22:09.874 INFO:teuthology.orchestra.run.smithi073.stdout:4254 punch from 0x55d51e6 to 0x55dd020, (0x7e3a bytes) 2024-09-21T14:22:09.887 INFO:teuthology.orchestra.run.smithi073.stdout:4255 trunc from 0xccb5513 to 0x196e28f 2024-09-21T14:22:09.904 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:09.905 INFO:teuthology.orchestra.run.smithi073.stdout:4257 read 0x7f0d8a thru 0x7f61f5 (0x546c bytes) 2024-09-21T14:22:09.912 INFO:teuthology.orchestra.run.smithi073.stdout:4259 write 0x2564d4a thru 0x2573db1 (0xf068 bytes) 2024-09-21T14:22:09.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:09.920 INFO:teuthology.orchestra.run.smithi073.stdout:4260 trunc from 0x2573db2 to 0x69b6d17 2024-09-21T14:22:09.934 INFO:teuthology.orchestra.run.smithi073.stdout:4261 write 0xa131b8b thru 0xa1349e3 (0x2e59 bytes) 2024-09-21T14:22:09.934 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:09.934+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:09.946 INFO:teuthology.orchestra.run.smithi073.stdout:4262 trunc from 0xa1349e4 to 0x213d5e5 2024-09-21T14:22:09.956 INFO:teuthology.orchestra.run.smithi073.stdout:4263 read 0x1917345 thru 0x19201d8 (0x8e94 bytes) 2024-09-21T14:22:09.973 INFO:teuthology.orchestra.run.smithi073.stdout:4264 trunc from 0x213d5e5 to 0xdafd526 2024-09-21T14:22:09.977 INFO:teuthology.orchestra.run.smithi073.stdout:4265 write 0x36f3698 thru 0x36f638a (0x2cf3 bytes) 2024-09-21T14:22:09.980 INFO:teuthology.orchestra.run.smithi073.stdout:4267 read 0x760d49c thru 0x7614cf9 (0x785e bytes) 2024-09-21T14:22:09.980 INFO:teuthology.orchestra.run.smithi073.stdout:4268 read 0x8d6fbcd thru 0x8d79e9f (0xa2d3 bytes) 2024-09-21T14:22:09.980 INFO:teuthology.orchestra.run.smithi073.stdout:4269 punch from 0x48694e to 0x48a213, (0x38c5 bytes) 2024-09-21T14:22:09.988 INFO:teuthology.orchestra.run.smithi073.stdout:4270 trunc from 0xdafd526 to 0x68b191e 2024-09-21T14:22:09.993 INFO:teuthology.orchestra.run.smithi073.stdout:4272 read 0x61f89da thru 0x620307f (0xa6a6 bytes) 2024-09-21T14:22:10.005 INFO:teuthology.orchestra.run.smithi073.stdout:4273 write 0xc2b51d7 thru 0xc2bc342 (0x716c bytes) 2024-09-21T14:22:10.008 INFO:teuthology.orchestra.run.smithi073.stdout:4275 writesame 0x292018c thru 0x2920a2d data_size 0x1(0x8a2 bytes) 2024-09-21T14:22:10.011 INFO:teuthology.orchestra.run.smithi073.stdout:4276 punch from 0x2df6bdc to 0x2df7d00, (0x1124 bytes) 2024-09-21T14:22:10.011 INFO:teuthology.orchestra.run.smithi073.stdout:4279 write 0x8a2034a thru 0x8a25819 (0x54d0 bytes) 2024-09-21T14:22:10.014 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.014 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.014 INFO:teuthology.orchestra.run.smithi073.stdout:4283 read 0x732fa91 thru 0x733b722 (0xbc92 bytes) 2024-09-21T14:22:10.015 INFO:teuthology.orchestra.run.smithi073.stdout:4284 trunc from 0xc2bc343 to 0xbd90db7 2024-09-21T14:22:10.020 INFO:teuthology.orchestra.run.smithi073.stdout:4285 read 0x5462ad1 thru 0x546e8b9 (0xbde9 bytes) 2024-09-21T14:22:10.021 INFO:teuthology.orchestra.run.smithi073.stdout:4286 read 0x4ea87ce thru 0x4eb35bb (0xadee bytes) 2024-09-21T14:22:10.021 INFO:teuthology.orchestra.run.smithi073.stdout:4287 read 0x9ee3722 thru 0x9ee4a09 (0x12e8 bytes) 2024-09-21T14:22:10.021 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.021 INFO:teuthology.orchestra.run.smithi073.stdout:4289 write 0x5bb5ecd thru 0x5bbad95 (0x4ec9 bytes) 2024-09-21T14:22:10.024 INFO:teuthology.orchestra.run.smithi073.stdout:4290 read 0x36a6ffe thru 0x36b00ff (0x9102 bytes) 2024-09-21T14:22:10.024 INFO:teuthology.orchestra.run.smithi073.stdout:4291 read 0x8f47e22 thru 0x8f4b834 (0x3a13 bytes) 2024-09-21T14:22:10.024 INFO:teuthology.orchestra.run.smithi073.stdout:4292 writesame 0x550415c thru 0x5513778 data_size 0x1(0xf61d bytes) 2024-09-21T14:22:10.025 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:10.024+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:10.046 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.503613692000002, 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43094 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658087526999999, 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355065000000002, 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43094 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387470+0000", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5480000000000003e-06 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387475+0000", 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1560000000000003e-06 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387479+0000", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7330000000000001e-06 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427814+0000", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040335226000000002 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427821+0000", 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6889999999999998e-06 2024-09-21T14:22:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43095 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658070752, 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040366234000000001, 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43095 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387484+0000", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.857e-06 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387489+0000", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387493+0000", 2024-09-21T14:22:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2830000000000003e-06 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427842+0000", 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040348342000000002 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427849+0000", 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9759999999999998e-06 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658056199000001, 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:22:10.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658037329999999, 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.069 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658023298, 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:10.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.658007299000001, 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657992403000002, 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:10.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657977466999998, 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:10.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.085 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657962486999999, 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:10.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657947871000001, 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:10.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657932824, 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:10.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657918264999999, 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:10.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:10.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.65790354, 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657889099999998, 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.106 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:10.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657874750000001, 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.110 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657854684, 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657839773999999, 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:10.121 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657824833999999, 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.125 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.657805141000001, 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:10.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:10.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:10.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:10.129 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:10.130 INFO:teuthology.orchestra.run.smithi073.stdout:4293 read 0x615bf35 thru 0x615d79d (0x1869 bytes) 2024-09-21T14:22:10.130 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.130 INFO:teuthology.orchestra.run.smithi073.stdout:4295 read 0x7ab948d thru 0x7ac69d1 (0xd545 bytes) 2024-09-21T14:22:10.130 INFO:teuthology.orchestra.run.smithi073.stdout:4296 write 0x50c7ed7 thru 0x50d4cfe (0xce28 bytes) 2024-09-21T14:22:10.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:10.133 INFO:teuthology.orchestra.run.smithi073.stdout:4297 read 0x7383746 thru 0x7391d18 (0xe5d3 bytes) 2024-09-21T14:22:10.133 INFO:teuthology.orchestra.run.smithi073.stdout:4298 write 0x5050c thru 0x5cb8c (0xc681 bytes) 2024-09-21T14:22:10.133 INFO:teuthology.orchestra.run.smithi073.stdout:4300 read 0x2ae9c35 thru 0x2aedd69 (0x4135 bytes) 2024-09-21T14:22:10.133 INFO:teuthology.orchestra.run.smithi073.stdout:4301 writesame 0xb567236 thru 0xb56a2d7 data_size 0x32(0x30a2 bytes) 2024-09-21T14:22:10.133 INFO:teuthology.orchestra.run.smithi073.stdout:4304 write 0x37d92ff thru 0x37e5a95 (0xc797 bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4305 writesame 0x5c7e827 thru 0x5c860a5 data_size 0x1(0x787f bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4306 write 0x40546e6 thru 0x4055e90 (0x17ab bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4308 write 0x895d53f thru 0x895dbf9 (0x6bb bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4310 read 0x769b1f4 thru 0x76a65a8 (0xb3b5 bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4311 write 0x59a86d3 thru 0x59b651e (0xde4c bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4314 write 0x227580d thru 0x2279b18 (0x430c bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4315 punch from 0x687724 to 0x691d3f, (0xa61b bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4316 read 0x39d0922 thru 0x39d2447 (0x1b26 bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4317 punch from 0x7a3ed7c to 0x7a467ca, (0x7a4e bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4318 write 0x72c174b thru 0x72c63bc (0x4c72 bytes) 2024-09-21T14:22:10.134 INFO:teuthology.orchestra.run.smithi073.stdout:4319 trunc from 0xbd90db7 to 0xe9dd059 2024-09-21T14:22:10.135 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.135 INFO:teuthology.orchestra.run.smithi073.stdout:4323 read 0xb95f0db thru 0xb96377b (0x46a1 bytes) 2024-09-21T14:22:10.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:10.129+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:10.142 INFO:teuthology.orchestra.run.smithi073.stdout:4327 trunc from 0xe9dd059 to 0x26620ac 2024-09-21T14:22:10.165 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.183 INFO:teuthology.orchestra.run.smithi073.stdout:4329 write 0xd6ce44c thru 0xd6da6e7 (0xc29c bytes) 2024-09-21T14:22:10.187 INFO:teuthology.orchestra.run.smithi073.stdout:4334 read 0x9198851 thru 0x919b41c (0x2bcc bytes) 2024-09-21T14:22:10.187 INFO:teuthology.orchestra.run.smithi073.stdout:4335 writesame 0xbbbe44e thru 0xbbcd99a data_size 0x1(0xf54d bytes) 2024-09-21T14:22:10.213 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.213 INFO:teuthology.orchestra.run.smithi073.stdout:4337 write 0xbb44c2a thru 0xbb50be0 (0xbfb7 bytes) 2024-09-21T14:22:10.214 INFO:teuthology.orchestra.run.smithi073.stdout:4338 read 0xa1599d0 thru 0xa15df1b (0x454c bytes) 2024-09-21T14:22:10.230 INFO:teuthology.orchestra.run.smithi073.stdout:4339 trunc from 0xd6da6e8 to 0x1f279e 2024-09-21T14:22:10.230 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:10.230+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:10.247 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:10.247 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:10.247 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:10.247 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:10.256 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:10.284 INFO:teuthology.orchestra.run.smithi073.stdout:4340 write 0x8b57c9e thru 0x8b62665 (0xa9c8 bytes) 2024-09-21T14:22:10.288 INFO:teuthology.orchestra.run.smithi073.stdout:4341 read 0x5937581 thru 0x5939bf3 (0x2673 bytes) 2024-09-21T14:22:10.288 INFO:teuthology.orchestra.run.smithi073.stdout:4342 write 0x3e8b918 thru 0x3e9ae08 (0xf4f1 bytes) 2024-09-21T14:22:10.291 INFO:teuthology.orchestra.run.smithi073.stdout:4343 punch from 0x576d637 to 0x577c6fa, (0xf0c3 bytes) 2024-09-21T14:22:10.300 INFO:teuthology.orchestra.run.smithi073.stdout:4346 write 0xbbd92c6 thru 0xbbe1bd8 (0x8913 bytes) 2024-09-21T14:22:10.304 INFO:teuthology.orchestra.run.smithi073.stdout:4348 read 0x16becdb thru 0x16c8d10 (0xa036 bytes) 2024-09-21T14:22:10.304 INFO:teuthology.orchestra.run.smithi073.stdout:4349 write 0x2778d49 thru 0x2788105 (0xf3bd bytes) 2024-09-21T14:22:10.307 INFO:teuthology.orchestra.run.smithi073.stdout:4350 write 0x2f1b345 thru 0x2f1ec3a (0x38f6 bytes) 2024-09-21T14:22:10.310 INFO:teuthology.orchestra.run.smithi073.stdout:4351 read 0x5e826c5 thru 0x5e842fa (0x1c36 bytes) 2024-09-21T14:22:10.310 INFO:teuthology.orchestra.run.smithi073.stdout:4352 write 0x7a51f51 thru 0x7a57d53 (0x5e03 bytes) 2024-09-21T14:22:10.318 INFO:teuthology.orchestra.run.smithi073.stdout:4353 trunc from 0xbbe1bd9 to 0x7892ccd 2024-09-21T14:22:10.330 INFO:teuthology.orchestra.run.smithi073.stdout:4356 trunc from 0x7892ccd to 0x674aef3 2024-09-21T14:22:10.338 INFO:teuthology.orchestra.run.smithi073.stdout:4357 writesame 0x3e6b928 thru 0x3e752a9 data_size 0x31(0x9982 bytes) 2024-09-21T14:22:10.341 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:10.340+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:10.342 INFO:teuthology.orchestra.run.smithi073.stdout:4358 write 0x72a7e3 thru 0x734e33 (0xa651 bytes) 2024-09-21T14:22:10.344 INFO:teuthology.orchestra.run.smithi073.stdout:4359 read 0x1523def thru 0x152b29a (0x74ac bytes) 2024-09-21T14:22:10.344 INFO:teuthology.orchestra.run.smithi073.stdout:4361 punch from 0x3f08088 to 0x3f0ae3c, (0x2db4 bytes) 2024-09-21T14:22:10.344 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.344 INFO:teuthology.orchestra.run.smithi073.stdout:4363 read 0x64d1fdc thru 0x64db175 (0x919a bytes) 2024-09-21T14:22:10.344 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.349 INFO:teuthology.orchestra.run.smithi073.stdout:4365 trunc from 0x674aef3 to 0xb7913a1 2024-09-21T14:22:10.356 INFO:teuthology.orchestra.run.smithi073.stdout:4367 trunc from 0xb7913a1 to 0xd48b3b3 2024-09-21T14:22:10.361 INFO:teuthology.orchestra.run.smithi073.stdout:4368 read 0x560381d thru 0x56090ba (0x589e bytes) 2024-09-21T14:22:10.361 INFO:teuthology.orchestra.run.smithi073.stdout:4370 write 0x10a5daf thru 0x10b1358 (0xb5aa bytes) 2024-09-21T14:22:10.365 INFO:teuthology.orchestra.run.smithi073.stdout:4372 read 0x445bb60 thru 0x445ec4f (0x30f0 bytes) 2024-09-21T14:22:10.365 INFO:teuthology.orchestra.run.smithi073.stdout:4375 punch from 0x83a9c9d to 0x83b46b8, (0xaa1b bytes) 2024-09-21T14:22:10.365 INFO:teuthology.orchestra.run.smithi073.stdout:4376 punch from 0x571e987 to 0x5721af4, (0x316d bytes) 2024-09-21T14:22:10.365 INFO:teuthology.orchestra.run.smithi073.stdout:4377 read 0xcf71feb thru 0xcf79d63 (0x7d79 bytes) 2024-09-21T14:22:10.365 INFO:teuthology.orchestra.run.smithi073.stdout:4378 write 0x2772bdd thru 0x2778db8 (0x61dc bytes) 2024-09-21T14:22:10.366 INFO:teuthology.orchestra.run.smithi073.stdout:4380 punch from 0x3712f4f to 0x371c768, (0x9819 bytes) 2024-09-21T14:22:10.366 INFO:teuthology.orchestra.run.smithi073.stdout:4383 write 0x882ee6f thru 0x8837ffd (0x918f bytes) 2024-09-21T14:22:10.369 INFO:teuthology.orchestra.run.smithi073.stdout:4384 writesame 0x8e0f2f8 thru 0x8e0f405 data_size 0x9(0x10e bytes) 2024-09-21T14:22:10.372 INFO:teuthology.orchestra.run.smithi073.stdout:4385 writesame 0x4723cb7 thru 0x4728644 data_size 0x23(0x498e bytes) 2024-09-21T14:22:10.380 INFO:teuthology.orchestra.run.smithi073.stdout:4386 trunc from 0xd48b3b3 to 0x8c9dfbd 2024-09-21T14:22:10.386 INFO:teuthology.orchestra.run.smithi073.stdout:4389 write 0x236fb8e thru 0x237a0cc (0xa53f bytes) 2024-09-21T14:22:10.392 INFO:teuthology.orchestra.run.smithi073.stdout:4390 punch from 0x4d4049b to 0x4d4c61e, (0xc183 bytes) 2024-09-21T14:22:10.397 INFO:teuthology.orchestra.run.smithi073.stdout:4391 trunc from 0x8c9dfbd to 0x4275c67 2024-09-21T14:22:10.406 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:10.407 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:10.407 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:10.407 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:10.407 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:10.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:4392 writesame 0xb5e193f thru 0xb5e199b data_size 0x1(0x5d bytes) 2024-09-21T14:22:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:4394 punch from 0x80038a9 to 0x800e9df, (0xb136 bytes) 2024-09-21T14:22:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:4398 read 0x9909873 thru 0x9917a9d (0xe22b bytes) 2024-09-21T14:22:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:4399 writesame 0x6848426 thru 0x6848612 data_size 0x1(0x1ed bytes) 2024-09-21T14:22:10.426 INFO:teuthology.orchestra.run.smithi073.stdout:4401 read 0x1278a5b thru 0x1280018 (0x75be bytes) 2024-09-21T14:22:10.426 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:10.426 INFO:teuthology.orchestra.run.smithi073.stdout:4403 writesame 0x65b59ad thru 0x65c19ab data_size 0x1(0xbfff bytes) 2024-09-21T14:22:10.432 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:10.432+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:10.448 INFO:teuthology.orchestra.run.smithi073.stdout:4404 read 0x67afa2a thru 0x67b6c4b (0x7222 bytes) 2024-09-21T14:22:10.448 INFO:teuthology.orchestra.run.smithi073.stdout:4407 writesame 0x5c74b7a thru 0x5c84618 data_size 0x1(0xfa9f bytes) 2024-09-21T14:22:10.461 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:22:10.481 INFO:teuthology.orchestra.run.smithi073.stdout:4408 write 0xd23baac thru 0xd23d86b (0x1dc0 bytes) 2024-09-21T14:22:10.484 INFO:teuthology.orchestra.run.smithi073.stdout:4409 clone 36 order 23 su 65536 sc 15 2024-09-21T14:22:10.533 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:10.532+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43741 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170248746999999, 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036952564, 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.561 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43741 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390184+0000", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390192+0000", 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.14e-06 2024-09-21T14:22:10.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390199+0000", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5150000000000003e-06 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390204+0000", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000001e-06 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.426948+0000", 2024-09-21T14:22:10.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036744128000000001 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427034+0000", 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6402000000000004e-05 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427137+0000", 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010275799999999999 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43742 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.17022394, 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037087785999999998, 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43742 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390209+0000", 2024-09-21T14:22:10.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390211+0000", 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1019999999999999e-06 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390216+0000", 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2870000000000004e-06 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390221+0000", 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427168+0000", 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036947520999999997 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427213+0000", 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4718000000000001e-05 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427297+0000", 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3939999999999999e-05 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43743 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170208387999999, 2024-09-21T14:22:10.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037293508000000003, 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43743 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390225+0000", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390224+0000", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390227+0000", 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1280000000000002e-06 2024-09-21T14:22:10.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390232+0000", 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5799999999999999e-06 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390237+0000", 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7890000000000002e-06 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427333+0000", 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037095876999999999 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6895000000000001e-05 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427518+0000", 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8611999999999997e-05 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43744 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170192197999999, 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037409745000000001, 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43744 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390241+0000", 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390243+0000", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390249+0000", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8239999999999996e-06 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390253+0000", 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7829999999999998e-06 2024-09-21T14:22:10.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427550+0000", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037296636000000001 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427608+0000", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8100000000000003e-05 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427651+0000", 2024-09-21T14:22:10.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2528999999999999e-05 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43745 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170175887999999, 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037596321000000002, 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.579 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43745 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390257+0000", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390259+0000", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0770000000000002e-06 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390264+0000", 2024-09-21T14:22:10.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3430000000000003e-06 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390268+0000", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0470000000000004e-06 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427691+0000", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037422596000000002 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427784+0000", 2024-09-21T14:22:10.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2780000000000003e-05 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427854+0000", 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9770000000000005e-05 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43746 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170161993000001, 2024-09-21T14:22:10.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037742629, 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43746 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390271+0000", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390273+0000", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1150000000000001e-06 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390279+0000", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6799999999999998e-06 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390283+0000", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427891+0000", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037607640999999997 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.427973+0000", 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.2170999999999996e-05 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428014+0000", 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0806000000000002e-05 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43747 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:10.587 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170113603000001, 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037837335999999999, 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43747 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:10.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390320+0000", 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390319+0000", 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390323+0000", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390332+0000", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9600000000000006e-06 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390339+0000", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8959999999999997e-06 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428048+0000", 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037709135999999997 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428117+0000", 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8534999999999993e-05 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428157+0000", 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0117000000000002e-05 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43748 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170088826000001, 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037945300000000001, 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43748 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390344+0000", 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390346+0000", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.464e-06 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390354+0000", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9619999999999993e-06 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390359+0000", 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428184+0000", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037824759999999999 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428230+0000", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6112000000000002e-05 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428290+0000", 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9327000000000003e-05 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43749 3.8 3:17e16120:::rbd_data.112168cdcfd1.000000000000000d:15 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170069130000002, 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038038324999999998, 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43749 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390364+0000", 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390368+0000", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1579999999999998e-06 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390375+0000", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390380+0000", 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9609999999999996e-06 2024-09-21T14:22:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428319+0000", 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.037939925999999999 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428364+0000", 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4623999999999998e-05 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428402+0000", 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43752 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170049340999999, 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038243006000000003, 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43752 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390384+0000", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390383+0000", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390386+0000", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0640000000000001e-06 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390392+0000", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4269999999999996e-06 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390397+0000", 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7650000000000001e-06 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428433+0000", 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038036164999999997 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428525+0000", 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1831999999999998e-05 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428627+0000", 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102099 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43753 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170032098, 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038351382000000003, 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43753 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390401+0000", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390403+0000", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0990000000000001e-06 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390410+0000", 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7299999999999999e-06 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390415+0000", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0730000000000004e-06 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428653+0000", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038237924 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428697+0000", 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4367999999999999e-05 2024-09-21T14:22:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428752+0000", 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5535000000000003e-05 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43791 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.170013913999998, 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038564714, 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43791 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390419+0000", 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390421+0000", 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.345e-06 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390428+0000", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6669999999999999e-06 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390433+0000", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.169e-06 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428791+0000", 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038358033 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428890+0000", 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9216999999999996e-05 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.428984+0000", 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3666999999999996e-05 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43792 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169995748000002, 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038678535, 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43792 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390437+0000", 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390444+0000", 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390451+0000", 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390460+0000", 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9150000000000008e-06 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429008+0000", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038547887000000003 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429052+0000", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3509e-05 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429116+0000", 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4078999999999994e-05 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43802 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169967239000002, 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038886423000000003, 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43802 2024-09-21T14:22:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390466+0000", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390468+0000", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.328e-06 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390476+0000", 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1939999999999998e-06 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390481+0000", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2449999999999998e-06 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429158+0000", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038676675000000001 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429249+0000", 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0657999999999998e-05 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429352+0000", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103729 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43803 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169947382, 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039086681999999998, 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43803 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390486+0000", 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390485+0000", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390488+0000", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1789999999999998e-06 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390494+0000", 2024-09-21T14:22:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3040000000000001e-06 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390499+0000", 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429393+0000", 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.038894038999999998 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429487+0000", 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4504000000000002e-05 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429572+0000", 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.509e-05 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43819 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169930045000001, 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039269071000000003, 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43819 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390503+0000", 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390505+0000", 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1490000000000001e-06 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390511+0000", 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0290000000000002e-06 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390516+0000", 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8060000000000002e-06 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429608+0000", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039092162 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429697+0000", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8712e-05 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429772+0000", 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5586000000000004e-05 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43820 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169687292999999, 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039230445000000003, 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43820 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390746+0000", 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390745+0000", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390749+0000", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.799e-06 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390770+0000", 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1035000000000001e-05 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390777+0000", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9890000000000003e-06 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039033209999999999 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429904+0000", 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4094999999999999e-05 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.429976+0000", 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1931e-05 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43832 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169650465, 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039328466999999999, 2024-09-21T14:22:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43832 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390783+0000", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390782+0000", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390785+0000", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8870000000000001e-06 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390792+0000", 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.951e-06 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390797+0000", 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1780000000000002e-06 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430003+0000", 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039205885000000003 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430046+0000", 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2867000000000001e-05 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430111+0000", 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5242999999999999e-05 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43833 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169631503000002, 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039492185999999999, 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43833 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390802+0000", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390801+0000", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390804+0000", 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390810+0000", 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4999999999999996e-06 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390815+0000", 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430137+0000", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039321964000000001 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430192+0000", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4787000000000002e-05 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430294+0000", 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000102186 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:43845 3.8 3:14042309:::rbd_data.1121bbf69350.000000000000000c:14 [read 151552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.169613780999999, 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039630430000000001, 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 43845 2024-09-21T14:22:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390819+0000", 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390821+0000", 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2689999999999998e-06 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390833+0000", 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2248e-05 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.390838+0000", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9540000000000003e-06 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430335+0000", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.039496297999999999 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430389+0000", 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4293999999999999e-05 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:47.430450+0000", 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0757999999999997e-05 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:10.647 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:10.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:10.649 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:10.648+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:10.749 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:10.749+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:10.791 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:10.792 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:10.792 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:10.792 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:10.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:10.850 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:10.849+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:10.902 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:10.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:10.903 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:10.903 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:10.903 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:10.911 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:10.952 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:10.952+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342927735000004, 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:22:11.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.057 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342916424999999, 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:11.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.061 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342901048999998, 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.062 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342883520999997, 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:22:11.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:22:11.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:22:11.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342864570000003, 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:22:11.070 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:22:11.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:11.074 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342848189999998, 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:11.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:22:11.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342838962000002, 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.081 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:11.051+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:11.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342827565999997, 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.085 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:11.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342804839000003, 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342788583000001, 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:22:11.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342773827999999, 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:22:11.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:22:11.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:22:11.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:22:11.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342764774000003, 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.103 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342753729999998, 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:22:11.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:22:11.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342738328999999, 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.111 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:11.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:22:11.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:22:11.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342721631000003, 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:22:11.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:11.119 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342706024000002, 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:22:11.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:22:11.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342688901999999, 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342670290999997, 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:22:11.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:22:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342650960999997, 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:22:11.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.342631324000003, 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.137 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:11.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:22:11.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.141 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.142 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.142 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:11.153 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:11.153+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:11.244 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:11.244 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.253 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:11.254 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:11.254+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:11.354 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:11.354+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:11.355 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.355 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:11.355 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:11.355 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:11.355 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.364 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:11.455 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:11.454+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:11.467 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:22:11.511 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.511 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:11.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:11.511 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079349716999999, 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:11.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:11.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079333075000001, 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:11.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:11.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079308025000003, 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079289873, 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:11.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:11.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079266636, 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:11.528 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:11.532 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079245569000001, 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:11.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:11.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079229208999998, 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079198929, 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079183790999998, 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079164194999997, 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079155417999999, 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.554 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:11.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:11.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079143057000003, 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:11.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:11.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079133640000002, 2024-09-21T14:22:11.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:11.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079112449, 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:11.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079095340999999, 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:11.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079078398, 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079062018999998, 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:11.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079020659000001, 2024-09-21T14:22:11.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.079000978000003, 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 18.674112530999999, 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:11.598 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.599 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone35 from 0xd23d86c (overlap 0x1f279e) to 0xf93cf 2024-09-21T14:22:11.600 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:11.599+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:11.600 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:11.667 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T14:22:11.700 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:11.699+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:11.708 INFO:teuthology.orchestra.run.smithi073.stdout:4410 trunc from 0xd23d86c to 0x8540f54 2024-09-21T14:22:11.737 INFO:teuthology.orchestra.run.smithi073.stdout:4411 read 0x7710531 thru 0x771c72a (0xc1fa bytes) 2024-09-21T14:22:11.737 INFO:teuthology.orchestra.run.smithi073.stdout:4412 writesame 0x4f7ac2c thru 0x4f88f3a data_size 0x1(0xe30f bytes) 2024-09-21T14:22:11.760 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.760 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:11.760 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:11.760 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.767 INFO:teuthology.orchestra.run.smithi073.stdout:4413 write 0x6ba5b5d thru 0x6ba872b (0x2bcf bytes) 2024-09-21T14:22:11.769 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:11.769 INFO:teuthology.orchestra.run.smithi073.stdout:4414 read 0x4d05b24 thru 0x4d146de (0xebbb bytes) 2024-09-21T14:22:11.770 INFO:teuthology.orchestra.run.smithi073.stdout:4415 punch from 0x6d9c75a to 0x6da7b1d, (0xb3c3 bytes) 2024-09-21T14:22:11.770 INFO:teuthology.orchestra.run.smithi073.stdout:4416 read 0x16d60ae thru 0x16d6397 (0x2ea bytes) 2024-09-21T14:22:11.778 INFO:teuthology.orchestra.run.smithi073.stdout:4417 write 0xaa29057 thru 0xaa36f08 (0xdeb2 bytes) 2024-09-21T14:22:11.780 INFO:teuthology.orchestra.run.smithi073.stdout:4418 punch from 0x67d2f77 to 0x67e1972, (0xe9fb bytes) 2024-09-21T14:22:11.780 INFO:teuthology.orchestra.run.smithi073.stdout:4419 punch from 0x68375f9 to 0x6839209, (0x1c10 bytes) 2024-09-21T14:22:11.780 INFO:teuthology.orchestra.run.smithi073.stdout:4420 write 0x6d5ca67 thru 0x6d68feb (0xc585 bytes) 2024-09-21T14:22:11.788 INFO:teuthology.orchestra.run.smithi073.stdout:4422 read 0x336926a thru 0x337434c (0xb0e3 bytes) 2024-09-21T14:22:11.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:11.796 INFO:teuthology.orchestra.run.smithi073.stdout:4423 write 0xc5118e1 thru 0xc518cdf (0x73ff bytes) 2024-09-21T14:22:11.798 INFO:teuthology.orchestra.run.smithi073.stdout:4425 write 0x93f231a thru 0x93fe592 (0xc279 bytes) 2024-09-21T14:22:11.800 INFO:teuthology.orchestra.run.smithi073.stdout:4427 write 0x81d61f6 thru 0x81d91fb (0x3006 bytes) 2024-09-21T14:22:11.801 INFO:teuthology.orchestra.run.smithi073.stdout:4429 write 0x9c02749 thru 0x9c089a9 (0x6261 bytes) 2024-09-21T14:22:11.802 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:11.810 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:11.809+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:11.814 INFO:teuthology.orchestra.run.smithi073.stdout:4432 trunc from 0xc518ce0 to 0x2251296 2024-09-21T14:22:11.859 INFO:teuthology.orchestra.run.smithi073.stdout:4433 read 0x7d498d thru 0x7d976d (0x4de1 bytes) 2024-09-21T14:22:11.859 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:11.865 INFO:teuthology.orchestra.run.smithi073.stdout:4435 trunc from 0x2251296 to 0x73b64ad 2024-09-21T14:22:11.870 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:11.885 INFO:teuthology.orchestra.run.smithi073.stdout:4437 writesame 0xed975ac thru 0xeda6a1d data_size 0x43(0xf472 bytes) 2024-09-21T14:22:11.885 INFO:teuthology.orchestra.run.smithi073.stdout:4438 read 0xb2804d5 thru 0xb2876e6 (0x7212 bytes) 2024-09-21T14:22:11.885 INFO:teuthology.orchestra.run.smithi073.stdout:4439 read 0x4833373 thru 0x483ae82 (0x7b10 bytes) 2024-09-21T14:22:11.899 INFO:teuthology.orchestra.run.smithi073.stdout:4440 trunc from 0xeda6a1e to 0x2b2b62a 2024-09-21T14:22:11.901 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:11.901+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:11.913 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:11.916 INFO:teuthology.orchestra.run.smithi073.stdout:4443 trunc from 0x2b2b62a to 0x4d9a775 2024-09-21T14:22:11.921 INFO:teuthology.orchestra.run.smithi073.stdout:4444 punch from 0x337f3e6 to 0x338de2a, (0xea44 bytes) 2024-09-21T14:22:11.925 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:11.925 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:11.925 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:11.926 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:11.926 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:11.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:11.930 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:11.934 INFO:teuthology.orchestra.run.smithi073.stdout:4445 write 0xbae5222 thru 0xbae8045 (0x2e24 bytes) 2024-09-21T14:22:11.935 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:11.938 INFO:teuthology.orchestra.run.smithi073.stdout:4448 punch from 0x6344028 to 0x63441bb, (0x193 bytes) 2024-09-21T14:22:11.938 INFO:teuthology.orchestra.run.smithi073.stdout:4449 read 0x8bd51c5 thru 0x8bdccfd (0x7b39 bytes) 2024-09-21T14:22:11.938 INFO:teuthology.orchestra.run.smithi073.stdout:4450 read 0xf61028 thru 0xf6b8fa (0xa8d3 bytes) 2024-09-21T14:22:11.938 INFO:teuthology.orchestra.run.smithi073.stdout:4452 write 0x24b46ff thru 0x24bec7e (0xa580 bytes) 2024-09-21T14:22:11.944 INFO:teuthology.orchestra.run.smithi073.stdout:4453 trunc from 0xbae8046 to 0x78afa78 2024-09-21T14:22:11.947 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:11.947 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:11.950 INFO:teuthology.orchestra.run.smithi073.stdout:4454 write 0x77f4b90 thru 0x7800184 (0xb5f5 bytes) 2024-09-21T14:22:11.964 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:11.964 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:11.965 INFO:teuthology.orchestra.run.smithi073.stdout:4455 write 0xc7d3516 thru 0xc7e30c3 (0xfbae bytes) 2024-09-21T14:22:11.970 INFO:teuthology.orchestra.run.smithi073.stdout:4456 read 0xa7e38bb thru 0xa7e3ae2 (0x228 bytes) 2024-09-21T14:22:11.970 INFO:teuthology.orchestra.run.smithi073.stdout:4458 write 0x6f76ea1 thru 0x6f81126 (0xa286 bytes) 2024-09-21T14:22:11.971 INFO:teuthology.orchestra.run.smithi073.stdout:4459 read 0x3ab6ff3 thru 0x3ac6911 (0xf91f bytes) 2024-09-21T14:22:11.972 INFO:teuthology.orchestra.run.smithi073.stdout:4461 write 0x33c03c2 thru 0x33c19e3 (0x1622 bytes) 2024-09-21T14:22:11.974 INFO:teuthology.orchestra.run.smithi073.stdout:4463 trunc from 0xc7e30c4 to 0xc9775bf 2024-09-21T14:22:11.981 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:11.981 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:11.986 INFO:teuthology.orchestra.run.smithi073.stdout:4464 write 0xcaa1bd0 thru 0xcaa9737 (0x7b68 bytes) 2024-09-21T14:22:11.986 INFO:teuthology.orchestra.run.smithi073.stdout:4465 writesame 0x1e86b02 thru 0x1e933ee data_size 0x1(0xc8ed bytes) 2024-09-21T14:22:11.999 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:12.002 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:12.002+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:12.005 INFO:teuthology.orchestra.run.smithi073.stdout:4466 punch from 0xa62178c to 0xa622761, (0xfd5 bytes) 2024-09-21T14:22:12.005 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.012 INFO:teuthology.orchestra.run.smithi073.stdout:4469 write 0xe375470 thru 0xe378559 (0x30ea bytes) 2024-09-21T14:22:12.016 INFO:teuthology.orchestra.run.smithi073.stdout:4470 trunc from 0xe37855a to 0xe7a0037 2024-09-21T14:22:12.017 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:12.017 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:12.020 INFO:teuthology.orchestra.run.smithi073.stdout:4471 read 0x1c40493 thru 0x1c4a026 (0x9b94 bytes) 2024-09-21T14:22:12.021 INFO:teuthology.orchestra.run.smithi073.stdout:4472 write 0x56412c4 thru 0x564c99d (0xb6da bytes) 2024-09-21T14:22:12.022 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.023 INFO:teuthology.orchestra.run.smithi073.stdout:4475 write 0xb41c0dd thru 0xb4260de (0xa002 bytes) 2024-09-21T14:22:12.027 INFO:teuthology.orchestra.run.smithi073.stdout:4476 read 0x9c10fb8 thru 0x9c1d4a6 (0xc4ef bytes) 2024-09-21T14:22:12.027 INFO:teuthology.orchestra.run.smithi073.stdout:4477 read 0x9330dee thru 0x933f700 (0xe913 bytes) 2024-09-21T14:22:12.028 INFO:teuthology.orchestra.run.smithi073.stdout:4478 punch from 0x28b29d2 to 0x28b8e64, (0x6492 bytes) 2024-09-21T14:22:12.028 INFO:teuthology.orchestra.run.smithi073.stdout:4479 punch from 0xdb52159 to 0xdb616b8, (0xf55f bytes) 2024-09-21T14:22:12.028 INFO:teuthology.orchestra.run.smithi073.stdout:4480 writesame 0x55f933e thru 0x5608a1a data_size 0x1(0xf6dd bytes) 2024-09-21T14:22:12.034 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:12.035 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:12.046 INFO:teuthology.orchestra.run.smithi073.stdout:4481 read 0x5eb24d8 thru 0x5eb4812 (0x233b bytes) 2024-09-21T14:22:12.046 INFO:teuthology.orchestra.run.smithi073.stdout:4483 write 0xd2c8fc0 thru 0xd2cda6d (0x4aae bytes) 2024-09-21T14:22:12.049 INFO:teuthology.orchestra.run.smithi073.stdout:4486 punch from 0x91548c1 to 0x9155986, (0x10c5 bytes) 2024-09-21T14:22:12.050 INFO:teuthology.orchestra.run.smithi073.stdout:4487 write 0x33c696a thru 0x33d632b (0xf9c2 bytes) 2024-09-21T14:22:12.053 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:12.053 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:12.054 INFO:teuthology.orchestra.run.smithi073.stdout:4488 read 0xa29f016 thru 0xa2a33de (0x43c9 bytes) 2024-09-21T14:22:12.054 INFO:teuthology.orchestra.run.smithi073.stdout:4489 write 0xc623070 thru 0xc62494c (0x18dd bytes) 2024-09-21T14:22:12.055 INFO:teuthology.orchestra.run.smithi073.stdout:4490 write 0xabf6f24 thru 0xabf81c9 (0x12a6 bytes) 2024-09-21T14:22:12.058 INFO:teuthology.orchestra.run.smithi073.stdout:4491 write 0xd7f7b6f thru 0xd802df7 (0xb289 bytes) 2024-09-21T14:22:12.059 INFO:teuthology.orchestra.run.smithi073.stdout:4492 read 0xe261996 thru 0xe264b97 (0x3202 bytes) 2024-09-21T14:22:12.059 INFO:teuthology.orchestra.run.smithi073.stdout:4493 write 0x135b105 thru 0x1369bd5 (0xead1 bytes) 2024-09-21T14:22:12.064 INFO:teuthology.orchestra.run.smithi073.stdout:4494 write 0x7f546cf thru 0x7f5ca7a (0x83ac bytes) 2024-09-21T14:22:12.066 INFO:teuthology.orchestra.run.smithi073.stdout:4495 writesame 0x3581746 thru 0x358ffb5 data_size 0x8(0xe870 bytes) 2024-09-21T14:22:12.070 INFO:teuthology.orchestra.run.smithi073.stdout:4498 read 0xaab7d87 thru 0xaaba7d5 (0x2a4f bytes) 2024-09-21T14:22:12.070 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.070 INFO:teuthology.orchestra.run.smithi073.stdout:4501 read 0x18dd331 thru 0x18ddcf6 (0x9c6 bytes) 2024-09-21T14:22:12.071 INFO:teuthology.orchestra.run.smithi073.stdout:4502 punch from 0x1367dc5 to 0x1373f66, (0xc1a1 bytes) 2024-09-21T14:22:12.071 INFO:teuthology.orchestra.run.smithi073.stdout:4504 write 0xd230ac0 thru 0xd235c5e (0x519f bytes) 2024-09-21T14:22:12.076 INFO:teuthology.orchestra.run.smithi073.stdout:4505 trunc from 0xe7a0037 to 0xb28ed9f 2024-09-21T14:22:12.083 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.083 INFO:teuthology.orchestra.run.smithi073.stdout:4507 punch from 0x9ae55bf to 0x9aed3e9, (0x7e2a bytes) 2024-09-21T14:22:12.084 INFO:teuthology.orchestra.run.smithi073.stdout:4508 write 0x6039282 thru 0x6046722 (0xd4a1 bytes) 2024-09-21T14:22:12.085 INFO:teuthology.orchestra.run.smithi073.stdout:4509 read 0x1edfe0f thru 0x1ee3a2f (0x3c21 bytes) 2024-09-21T14:22:12.086 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.086 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.086 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.086 INFO:teuthology.orchestra.run.smithi073.stdout:4511 read 0x82f8efb thru 0x8306dee (0xdef4 bytes) 2024-09-21T14:22:12.087 INFO:teuthology.orchestra.run.smithi073.stdout:4512 read 0xa9dfbe0 thru 0xa9e4a36 (0x4e57 bytes) 2024-09-21T14:22:12.087 INFO:teuthology.orchestra.run.smithi073.stdout:4513 punch from 0x7de493 to 0x7eb8ad, (0xd41a bytes) 2024-09-21T14:22:12.087 INFO:teuthology.orchestra.run.smithi073.stdout:4515 write 0x6b6e419 thru 0x6b76ef4 (0x8adc bytes) 2024-09-21T14:22:12.088 INFO:teuthology.orchestra.run.smithi073.stdout:4516 write 0x46ab68a thru 0x46af046 (0x39bd bytes) 2024-09-21T14:22:12.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:12.097 INFO:teuthology.orchestra.run.smithi073.stdout:4517 write 0xd7b0e19 thru 0xd7b628f (0x5477 bytes) 2024-09-21T14:22:12.099 INFO:teuthology.orchestra.run.smithi073.stdout:4519 read 0xd2e4001 thru 0xd2f16ab (0xd6ab bytes) 2024-09-21T14:22:12.099 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.100 INFO:teuthology.orchestra.run.smithi073.stdout:4522 write 0xd35e7ad thru 0xd367f45 (0x9799 bytes) 2024-09-21T14:22:12.103 INFO:teuthology.orchestra.run.smithi073.stdout:4523 writesame 0x12b5582 thru 0x12c3cdc data_size 0x1(0xe75b bytes) 2024-09-21T14:22:12.103 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:12.103+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:12.123 INFO:teuthology.orchestra.run.smithi073.stdout:4524 write 0xc64ee95 thru 0xc65a8c7 (0xba33 bytes) 2024-09-21T14:22:12.126 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.126 INFO:teuthology.orchestra.run.smithi073.stdout:4526 writesame 0x33c76b1 thru 0x33c8721 data_size 0x17(0x1071 bytes) 2024-09-21T14:22:12.128 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.136 INFO:teuthology.orchestra.run.smithi073.stdout:4530 trunc from 0xd7b6290 to 0x6fc6e8d 2024-09-21T14:22:12.161 INFO:teuthology.orchestra.run.smithi073.stdout:4531 trunc from 0x6fc6e8d to 0xee731f 2024-09-21T14:22:12.172 INFO:teuthology.orchestra.run.smithi073.stdout:4533 trunc from 0xee731f to 0x2b3ed09 2024-09-21T14:22:12.178 INFO:teuthology.orchestra.run.smithi073.stdout:4538 read 0x171b699 thru 0x171dfc3 (0x292b bytes) 2024-09-21T14:22:12.196 INFO:teuthology.orchestra.run.smithi073.stdout:4539 write 0xcddba52 thru 0xcddcb5e (0x110d bytes) 2024-09-21T14:22:12.199 INFO:teuthology.orchestra.run.smithi073.stdout:4543 write 0x9176d85 thru 0x91840ad (0xd329 bytes) 2024-09-21T14:22:12.204 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:12.204+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:12.205 INFO:teuthology.orchestra.run.smithi073.stdout:4544 trunc from 0xcddcb5f to 0xed181c5 2024-09-21T14:22:12.209 INFO:teuthology.orchestra.run.smithi073.stdout:4546 read 0xf2ff84 thru 0xf3f219 (0xf296 bytes) 2024-09-21T14:22:12.210 INFO:teuthology.orchestra.run.smithi073.stdout:4547 flatten 2024-09-21T14:22:12.214 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.214 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.222 INFO:teuthology.orchestra.run.smithi073.stdout:4548 trunc from 0xed181c5 to 0xa14a3c5 2024-09-21T14:22:12.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:12.229 INFO:teuthology.orchestra.run.smithi073.stdout:4549 writesame 0x68f8105 thru 0x6904aed data_size 0x25(0xc9e9 bytes) 2024-09-21T14:22:12.231 INFO:teuthology.orchestra.run.smithi073.stdout:4551 read 0x1ba579e thru 0x1ba9220 (0x3a83 bytes) 2024-09-21T14:22:12.231 INFO:teuthology.orchestra.run.smithi073.stdout:4552 punch from 0x998be6 to 0x9a17d5, (0x8bef bytes) 2024-09-21T14:22:12.240 INFO:teuthology.orchestra.run.smithi073.stdout:4553 trunc from 0xa14a3c5 to 0x2c8f909 2024-09-21T14:22:12.265 INFO:teuthology.orchestra.run.smithi073.stdout:4554 write 0x7b36d14 thru 0x7b3bc48 (0x4f35 bytes) 2024-09-21T14:22:12.268 INFO:teuthology.orchestra.run.smithi073.stdout:4555 punch from 0x7a3c3f0 to 0x7a4a425, (0xe035 bytes) 2024-09-21T14:22:12.268 INFO:teuthology.orchestra.run.smithi073.stdout:4556 read 0x2984236 thru 0x298e6b9 (0xa484 bytes) 2024-09-21T14:22:12.269 INFO:teuthology.orchestra.run.smithi073.stdout:4557 read 0x4fee042 thru 0x4ff4c2a (0x6be9 bytes) 2024-09-21T14:22:12.269 INFO:teuthology.orchestra.run.smithi073.stdout:4558 write 0x54846d3 thru 0x5492149 (0xda77 bytes) 2024-09-21T14:22:12.271 INFO:teuthology.orchestra.run.smithi073.stdout:4560 punch from 0x41cf611 to 0x41cf804, (0x1f3 bytes) 2024-09-21T14:22:12.278 INFO:teuthology.orchestra.run.smithi073.stdout:4563 trunc from 0x7b3bc49 to 0xe14fbe3 2024-09-21T14:22:12.286 INFO:teuthology.orchestra.run.smithi073.stdout:4564 trunc from 0xe14fbe3 to 0xbefaf2b 2024-09-21T14:22:12.293 INFO:teuthology.orchestra.run.smithi073.stdout:4565 write 0x582a67d thru 0x583a62a (0xffae bytes) 2024-09-21T14:22:12.295 INFO:teuthology.orchestra.run.smithi073.stdout:4566 write 0xb92ca37 thru 0xb935549 (0x8b13 bytes) 2024-09-21T14:22:12.302 INFO:teuthology.orchestra.run.smithi073.stdout:4567 read 0x757570 thru 0x760e7e (0x990f bytes) 2024-09-21T14:22:12.305 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:12.305+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:12.309 INFO:teuthology.orchestra.run.smithi073.stdout:4568 trunc from 0xbefaf2b to 0x6a2b3a2 2024-09-21T14:22:12.333 INFO:teuthology.orchestra.run.smithi073.stdout:4571 writesame 0x942edc2 thru 0x943b6ca data_size 0x49(0xc909 bytes) 2024-09-21T14:22:12.340 INFO:teuthology.orchestra.run.smithi073.stdout:4573 trunc from 0x943b6cb to 0x359632d 2024-09-21T14:22:12.349 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.349 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.350 INFO:teuthology.orchestra.run.smithi073.stdout:4574 write 0x21284e3 thru 0x212f520 (0x703e bytes) 2024-09-21T14:22:12.352 INFO:teuthology.orchestra.run.smithi073.stdout:4575 read 0x2dea138 thru 0x2df46ec (0xa5b5 bytes) 2024-09-21T14:22:12.352 INFO:teuthology.orchestra.run.smithi073.stdout:4576 read 0x2053366 thru 0x2055d91 (0x2a2c bytes) 2024-09-21T14:22:12.357 INFO:teuthology.orchestra.run.smithi073.stdout:4577 trunc from 0x359632d to 0x2a6cbe 2024-09-21T14:22:12.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:12.367 INFO:teuthology.orchestra.run.smithi073.stdout:4578 trunc from 0x2a6cbe to 0xd61c44 2024-09-21T14:22:12.371 INFO:teuthology.orchestra.run.smithi073.stdout:4579 punch from 0x89e7b4 to 0x8abdf4, (0xd640 bytes) 2024-09-21T14:22:12.391 INFO:teuthology.orchestra.run.smithi073.stdout:4580 write 0xc7c6a8e thru 0xc7d0861 (0x9dd4 bytes) 2024-09-21T14:22:12.394 INFO:teuthology.orchestra.run.smithi073.stdout:4582 write 0x3cf4126 thru 0x3d02aaa (0xe985 bytes) 2024-09-21T14:22:12.396 INFO:teuthology.orchestra.run.smithi073.stdout:4584 write 0x7d59577 thru 0x7d5fdbf (0x6849 bytes) 2024-09-21T14:22:12.398 INFO:teuthology.orchestra.run.smithi073.stdout:4585 read 0x2bc14cf thru 0x2bcd878 (0xc3aa bytes) 2024-09-21T14:22:12.399 INFO:teuthology.orchestra.run.smithi073.stdout:4586 writesame 0x8c02719 thru 0x8c02f16 data_size 0x16(0x7fe bytes) 2024-09-21T14:22:12.402 INFO:teuthology.orchestra.run.smithi073.stdout:4587 punch from 0x589b48b to 0x589cd11, (0x1886 bytes) 2024-09-21T14:22:12.402 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.402 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.402 INFO:teuthology.orchestra.run.smithi073.stdout:4593 write 0x839fd60 thru 0x83a4992 (0x4c33 bytes) 2024-09-21T14:22:12.405 INFO:teuthology.orchestra.run.smithi073.stdout:4594 writesame 0x9446fea thru 0x9449029 data_size 0x40(0x2040 bytes) 2024-09-21T14:22:12.406 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:12.405+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:12.407 INFO:teuthology.orchestra.run.smithi073.stdout:4595 write 0x326085 thru 0x329230 (0x31ac bytes) 2024-09-21T14:22:12.411 INFO:teuthology.orchestra.run.smithi073.stdout:4596 trunc from 0xc7d0862 to 0xa34e944 2024-09-21T14:22:12.418 INFO:teuthology.orchestra.run.smithi073.stdout:4597 write 0x3a33397 thru 0x3a371a8 (0x3e12 bytes) 2024-09-21T14:22:12.419 INFO:teuthology.orchestra.run.smithi073.stdout:4598 punch from 0x3e0f110 to 0x3e1c1c2, (0xd0b2 bytes) 2024-09-21T14:22:12.419 INFO:teuthology.orchestra.run.smithi073.stdout:4599 read 0x7d981ba thru 0x7da7643 (0xf48a bytes) 2024-09-21T14:22:12.419 INFO:teuthology.orchestra.run.smithi073.stdout:4600 writesame 0x8830a95 thru 0x883b69e data_size 0x3d(0xac0a bytes) 2024-09-21T14:22:12.422 INFO:teuthology.orchestra.run.smithi073.stdout:4601 read 0x72e865 thru 0x733500 (0x4c9c bytes) 2024-09-21T14:22:12.422 INFO:teuthology.orchestra.run.smithi073.stdout:4602 write 0x3424854 thru 0x342e794 (0x9f41 bytes) 2024-09-21T14:22:12.423 INFO:teuthology.orchestra.run.smithi073.stdout:4604 read 0x5105d27 thru 0x510912b (0x3405 bytes) 2024-09-21T14:22:12.424 INFO:teuthology.orchestra.run.smithi073.stdout:4605 punch from 0x502e22a to 0x5030e5a, (0x2c30 bytes) 2024-09-21T14:22:12.424 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.424 INFO:teuthology.orchestra.run.smithi073.stdout:4609 writesame 0x50fa901 thru 0x50ff852 data_size 0x47(0x4f52 bytes) 2024-09-21T14:22:12.425 INFO:teuthology.orchestra.run.smithi073.stdout:4610 write 0x72c31e9 thru 0x72c78e5 (0x46fd bytes) 2024-09-21T14:22:12.427 INFO:teuthology.orchestra.run.smithi073.stdout:4612 write 0x5fa68fb thru 0x5fb09f2 (0xa0f8 bytes) 2024-09-21T14:22:12.432 INFO:teuthology.orchestra.run.smithi073.stdout:4614 punch from 0x859e605 to 0x85a715d, (0x8b58 bytes) 2024-09-21T14:22:12.432 INFO:teuthology.orchestra.run.smithi073.stdout:4615 read 0x9178695 thru 0x917f611 (0x6f7d bytes) 2024-09-21T14:22:12.433 INFO:teuthology.orchestra.run.smithi073.stdout:4616 write 0x1e1e577 thru 0x1e2691b (0x83a5 bytes) 2024-09-21T14:22:12.436 INFO:teuthology.orchestra.run.smithi073.stdout:4617 write 0x344e610 thru 0x34521d7 (0x3bc8 bytes) 2024-09-21T14:22:12.449 INFO:teuthology.orchestra.run.smithi073.stdout:4618 trunc from 0xa34e944 to 0x5b02b5 2024-09-21T14:22:12.478 INFO:teuthology.orchestra.run.smithi073.stdout:4619 writesame 0x66e9b6c thru 0x66f21fa data_size 0x31(0x868f bytes) 2024-09-21T14:22:12.479 INFO:teuthology.orchestra.run.smithi073.stdout:4620 write 0x377b844 thru 0x377f882 (0x403f bytes) 2024-09-21T14:22:12.481 INFO:teuthology.orchestra.run.smithi073.stdout:4622 write 0x19516c7 thru 0x1953a9d (0x23d7 bytes) 2024-09-21T14:22:12.484 INFO:teuthology.orchestra.run.smithi073.stdout:4626 trunc from 0x66f21fb to 0x51d9c2c 2024-09-21T14:22:12.486 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.486 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:12.507 INFO:teuthology.orchestra.run.smithi073.stdout:4627 write 0xbff12b0 thru 0xbffb477 (0xa1c8 bytes) 2024-09-21T14:22:12.507 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:12.507+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:12.508 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:12.508 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:12.508 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-21T14:22:12.508 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:22:12.508 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:22:12.508 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-21T14:22:12.510 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.510 INFO:teuthology.orchestra.run.smithi073.stdout:4631 write 0x4683479 thru 0x4692e22 (0xf9aa bytes) 2024-09-21T14:22:12.512 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:12.512 INFO:teuthology.orchestra.run.smithi073.stdout:4633 write 0x3992b72 thru 0x3996770 (0x3bff bytes) 2024-09-21T14:22:12.513 INFO:teuthology.orchestra.run.smithi073.stdout:4634 read 0x312ec51 thru 0x313d50a (0xe8ba bytes) 2024-09-21T14:22:12.513 INFO:teuthology.orchestra.run.smithi073.stdout:4636 read 0x5b1155a thru 0x5b125c8 (0x106f bytes) 2024-09-21T14:22:12.515 INFO:teuthology.orchestra.run.smithi073.stdout:4638 trunc from 0xbffb478 to 0xcf1bb8a 2024-09-21T14:22:12.520 INFO:teuthology.orchestra.run.smithi073.stdout:4639 clone 37 order 22 su 65536 sc 15 2024-09-21T14:22:12.608 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:12.607+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:12.609 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.609 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.610 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:12.610 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:22:12.610 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:12.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:12.710 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:12.709+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:12.759 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.759 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:12.812 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:12.812+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:12.870 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:12.870 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:12.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:12.911 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:12.910+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:13.011 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:13.011+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:13.021 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.021 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:13.114 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:13.114+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:13.173 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.173 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:13.223 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:13.223+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:13.315 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:13.315+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:13.323 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.323 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:13.416 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:13.416+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:13.475 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.475 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:22:13.484 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:13.517 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:13.516+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:13.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:13.618+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:13.633 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.633 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:13.701 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T14:22:13.706 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:13.706 INFO:teuthology.orchestra.run.smithi073.stdout:4641 read 0x4a090b3 thru 0x4a11624 (0x8572 bytes) 2024-09-21T14:22:13.707 INFO:teuthology.orchestra.run.smithi073.stdout:4642 writesame 0x544a610 thru 0x544d519 data_size 0x12(0x2f0a bytes) 2024-09-21T14:22:13.714 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:13.714 INFO:teuthology.orchestra.run.smithi073.stdout:4644 read 0x6f26e7f thru 0x6f2f55e (0x86e0 bytes) 2024-09-21T14:22:13.715 INFO:teuthology.orchestra.run.smithi073.stdout:4646 read 0x54a11f9 thru 0x54ab1e6 (0x9fee bytes) 2024-09-21T14:22:13.715 INFO:teuthology.orchestra.run.smithi073.stdout:4648 punch from 0xc6f6287 to 0xc6f9095, (0x2e0e bytes) 2024-09-21T14:22:13.715 INFO:teuthology.orchestra.run.smithi073.stdout:4650 writesame 0x94a5045 thru 0x94b2b37 data_size 0x1(0xdaf3 bytes) 2024-09-21T14:22:13.719 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:13.718+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:13.744 INFO:teuthology.orchestra.run.smithi073.stdout:4651 write 0xed03e57 thru 0xed05127 (0x12d1 bytes) 2024-09-21T14:22:13.751 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.751 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.763 INFO:teuthology.orchestra.run.smithi073.stdout:4652 trunc from 0xed05128 to 0x24eb45d 2024-09-21T14:22:13.764 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:13.788 INFO:teuthology.orchestra.run.smithi073.stdout:4653 trunc from 0x24eb45d to 0x21bf9ec 2024-09-21T14:22:13.803 INFO:teuthology.orchestra.run.smithi073.stdout:4654 trunc from 0x21bf9ec to 0x7346047 2024-09-21T14:22:13.811 INFO:teuthology.orchestra.run.smithi073.stdout:4655 trunc from 0x7346047 to 0x914061e 2024-09-21T14:22:13.815 INFO:teuthology.orchestra.run.smithi073.stdout:4656 write 0x8970e7d thru 0x89731be (0x2342 bytes) 2024-09-21T14:22:13.818 INFO:teuthology.orchestra.run.smithi073.stdout:4657 write 0x32fe282 thru 0x33095c4 (0xb343 bytes) 2024-09-21T14:22:13.820 INFO:teuthology.orchestra.run.smithi073.stdout:4658 read 0x696fc42 thru 0x6979f23 (0xa2e2 bytes) 2024-09-21T14:22:13.820 INFO:teuthology.orchestra.run.smithi073.stdout:4659 writesame 0x6315ed8 thru 0x6319efd data_size 0x45(0x4026 bytes) 2024-09-21T14:22:13.820 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:13.820+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:13.822 INFO:teuthology.orchestra.run.smithi073.stdout:4660 write 0x4b60f42 thru 0x4b6b19a (0xa259 bytes) 2024-09-21T14:22:13.832 INFO:teuthology.orchestra.run.smithi073.stdout:4662 trunc from 0x914061e to 0x3e30375 2024-09-21T14:22:13.843 INFO:teuthology.orchestra.run.smithi073.stdout:4665 punch from 0x1c57f03 to 0x1c63200, (0xb2fd bytes) 2024-09-21T14:22:13.852 INFO:teuthology.orchestra.run.smithi073.stdout:4666 trunc from 0x3e30375 to 0xbd2b07c 2024-09-21T14:22:13.857 INFO:teuthology.orchestra.run.smithi073.stdout:4667 read 0x9dcbfd0 thru 0x9dcfd37 (0x3d68 bytes) 2024-09-21T14:22:13.862 INFO:teuthology.orchestra.run.smithi073.stdout:4669 write 0xc99afe4 thru 0xc9a4682 (0x969f bytes) 2024-09-21T14:22:13.866 INFO:teuthology.orchestra.run.smithi073.stdout:4670 write 0x26c6c6 thru 0x277803 (0xb13e bytes) 2024-09-21T14:22:13.868 INFO:teuthology.orchestra.run.smithi073.stdout:4671 write 0xc980116 thru 0xc9884ea (0x83d5 bytes) 2024-09-21T14:22:13.871 INFO:teuthology.orchestra.run.smithi073.stdout:4672 write 0xa9c147a thru 0xa9d12b7 (0xfe3e bytes) 2024-09-21T14:22:13.874 INFO:teuthology.orchestra.run.smithi073.stdout:4673 write 0xa9263b thru 0xa9272e (0xf4 bytes) 2024-09-21T14:22:13.875 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:13.876 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.876 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:13.885 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:13.890 INFO:teuthology.orchestra.run.smithi073.stdout:4677 trunc from 0xc9a4683 to 0xa03cb5 2024-09-21T14:22:13.924 INFO:teuthology.orchestra.run.smithi073.stdout:4681 write 0xca2c7c6 thru 0xca30aab (0x42e6 bytes) 2024-09-21T14:22:13.928 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:13.928+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:13.929 INFO:teuthology.orchestra.run.smithi073.stdout:4682 write 0x76fad39 thru 0x7707a76 (0xcd3e bytes) 2024-09-21T14:22:13.933 INFO:teuthology.orchestra.run.smithi073.stdout:4683 read 0xa043be2 thru 0xa053a80 (0xfe9f bytes) 2024-09-21T14:22:13.933 INFO:teuthology.orchestra.run.smithi073.stdout:4684 write 0x2b69416 thru 0x2b7932e (0xff19 bytes) 2024-09-21T14:22:13.935 INFO:teuthology.orchestra.run.smithi073.stdout:4685 write 0x3776a3e thru 0x3782061 (0xb624 bytes) 2024-09-21T14:22:13.937 INFO:teuthology.orchestra.run.smithi073.stdout:4686 writesame 0xb9b2ba9 thru 0xb9b93ad data_size 0x1(0x6805 bytes) 2024-09-21T14:22:13.952 INFO:teuthology.orchestra.run.smithi073.stdout:4688 read 0x1c3601d thru 0x1c38aa8 (0x2a8c bytes) 2024-09-21T14:22:13.952 INFO:teuthology.orchestra.run.smithi073.stdout:4689 read 0xf89d7c thru 0xf909b8 (0x6c3d bytes) 2024-09-21T14:22:13.952 INFO:teuthology.orchestra.run.smithi073.stdout:4690 write 0x3fc1d64 thru 0x3fc545a (0x36f7 bytes) 2024-09-21T14:22:13.955 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:13.962 INFO:teuthology.orchestra.run.smithi073.stdout:4692 write 0xcff06fa thru 0xcffd6e5 (0xcfec bytes) 2024-09-21T14:22:13.974 INFO:teuthology.orchestra.run.smithi073.stdout:4693 write 0xe198c0b thru 0xe1a3dbf (0xb1b5 bytes) 2024-09-21T14:22:13.976 INFO:teuthology.orchestra.run.smithi073.stdout:4694 write 0x4aa67b4 thru 0x4aa6829 (0x76 bytes) 2024-09-21T14:22:13.980 INFO:teuthology.orchestra.run.smithi073.stdout:4695 trunc from 0xe1a3dc0 to 0xcc62112 2024-09-21T14:22:13.987 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:13.987 INFO:teuthology.orchestra.run.smithi073.stdout:4697 read 0x80f23ce thru 0x80f5f36 (0x3b69 bytes) 2024-09-21T14:22:13.987 INFO:teuthology.orchestra.run.smithi073.stdout:4698 read 0xa09f896 thru 0xa0a1887 (0x1ff2 bytes) 2024-09-21T14:22:13.995 INFO:teuthology.orchestra.run.smithi073.stdout:4699 writesame 0xee1bb0c thru 0xee246f8 data_size 0x1(0x8bed bytes) 2024-09-21T14:22:13.995 INFO:teuthology.orchestra.run.smithi073.stdout:4701 punch from 0xbbfc773 to 0xbc05a57, (0x92e4 bytes) 2024-09-21T14:22:13.995 INFO:teuthology.orchestra.run.smithi073.stdout:4702 write 0x557acb7 thru 0x5580469 (0x57b3 bytes) 2024-09-21T14:22:13.996 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:13.996 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:14.005 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:14.009 INFO:teuthology.orchestra.run.smithi073.stdout:4703 trunc from 0xee246f9 to 0x56456c5 2024-09-21T14:22:14.021 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:14.021+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:14.030 INFO:teuthology.orchestra.run.smithi073.stdout:4704 trunc from 0x56456c5 to 0x16c4f1 2024-09-21T14:22:14.046 INFO:teuthology.orchestra.run.smithi073.stdout:4706 read 0x154e75 thru 0x15e833 (0x99bf bytes) 2024-09-21T14:22:14.047 INFO:teuthology.orchestra.run.smithi073.stdout:4707 read 0x7f362 thru 0x86736 (0x73d5 bytes) 2024-09-21T14:22:14.061 INFO:teuthology.orchestra.run.smithi073.stdout:4708 writesame 0x7691e32 thru 0x769cdf9 data_size 0x14(0xafc8 bytes) 2024-09-21T14:22:14.061 INFO:teuthology.orchestra.run.smithi073.stdout:4711 read 0x26fe463 thru 0x2707649 (0x91e7 bytes) 2024-09-21T14:22:14.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:14.070 INFO:teuthology.orchestra.run.smithi073.stdout:4712 writesame 0xaa3115e thru 0xaa39b08 data_size 0x1(0x89ab bytes) 2024-09-21T14:22:14.070 INFO:teuthology.orchestra.run.smithi073.stdout:4713 read 0x8462151 thru 0x846af0c (0x8dbc bytes) 2024-09-21T14:22:14.070 INFO:teuthology.orchestra.run.smithi073.stdout:4714 read 0x36ed358 thru 0x36f13ca (0x4073 bytes) 2024-09-21T14:22:14.071 INFO:teuthology.orchestra.run.smithi073.stdout:4715 punch from 0x731dff0 to 0x73292be, (0xb2ce bytes) 2024-09-21T14:22:14.071 INFO:teuthology.orchestra.run.smithi073.stdout:4716 read 0x381fd68 thru 0x382cf16 (0xd1af bytes) 2024-09-21T14:22:14.078 INFO:teuthology.orchestra.run.smithi073.stdout:4717 write 0xc758466 thru 0xc762fed (0xab88 bytes) 2024-09-21T14:22:14.082 INFO:teuthology.orchestra.run.smithi073.stdout:4718 read 0x520f1cc thru 0x5213986 (0x47bb bytes) 2024-09-21T14:22:14.082 INFO:teuthology.orchestra.run.smithi073.stdout:4719 write 0x920877e thru 0x921820d (0xfa90 bytes) 2024-09-21T14:22:14.085 INFO:teuthology.orchestra.run.smithi073.stdout:4722 read 0x7f51916 thru 0x7f544fa (0x2be5 bytes) 2024-09-21T14:22:14.086 INFO:teuthology.orchestra.run.smithi073.stdout:4723 write 0x6387ef5 thru 0x63953da (0xd4e6 bytes) 2024-09-21T14:22:14.093 INFO:teuthology.orchestra.run.smithi073.stdout:4725 trunc from 0xc762fee to 0x91d27bf 2024-09-21T14:22:14.103 INFO:teuthology.orchestra.run.smithi073.stdout:4727 write 0x289ec8d thru 0x28a5f94 (0x7308 bytes) 2024-09-21T14:22:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:4728 punch from 0x4d1dade to 0x4d2068f, (0x2bb1 bytes) 2024-09-21T14:22:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:4729 read 0x609cd82 thru 0x60a9812 (0xca91 bytes) 2024-09-21T14:22:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:4730 read 0x6f181ae thru 0x6f25ad4 (0xd927 bytes) 2024-09-21T14:22:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:4733 read 0x1af08f4 thru 0x1af8cc3 (0x83d0 bytes) 2024-09-21T14:22:14.114 INFO:teuthology.orchestra.run.smithi073.stdout:4734 write 0xc712725 thru 0xc71d30a (0xabe6 bytes) 2024-09-21T14:22:14.117 INFO:teuthology.orchestra.run.smithi073.stdout:4735 write 0x7831d9e thru 0x7840264 (0xe4c7 bytes) 2024-09-21T14:22:14.122 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:14.122+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:14.128 INFO:teuthology.orchestra.run.smithi073.stdout:4737 trunc from 0xc71d30b to 0x9bfc6ff 2024-09-21T14:22:14.149 INFO:teuthology.orchestra.run.smithi073.stdout:4738 writesame 0xe647a0c thru 0xe64f375 data_size 0x1(0x796a bytes) 2024-09-21T14:22:14.149 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.149 INFO:teuthology.orchestra.run.smithi073.stdout:4742 write 0x10b232b thru 0x10c14a8 (0xf17e bytes) 2024-09-21T14:22:14.151 INFO:teuthology.orchestra.run.smithi073.stdout:4743 writesame 0x1d76cc4 thru 0x1d7a2f9 data_size 0x1b(0x3636 bytes) 2024-09-21T14:22:14.153 INFO:teuthology.orchestra.run.smithi073.stdout:4744 write 0x84f582b thru 0x85041eb (0xe9c1 bytes) 2024-09-21T14:22:14.156 INFO:teuthology.orchestra.run.smithi073.stdout:4747 punch from 0xad440 to 0xad68e, (0x24e bytes) 2024-09-21T14:22:14.156 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.157 INFO:teuthology.orchestra.run.smithi073.stdout:4750 writesame 0xb965424 thru 0xb9690f0 data_size 0xb(0x3ccd bytes) 2024-09-21T14:22:14.157 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:14.157 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:14.159 INFO:teuthology.orchestra.run.smithi073.stdout:4751 read 0x5264ba thru 0x526d04 (0x84b bytes) 2024-09-21T14:22:14.163 INFO:teuthology.orchestra.run.smithi073.stdout:4752 trunc from 0xe64f376 to 0xb2e4762 2024-09-21T14:22:14.166 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:14.179 INFO:teuthology.orchestra.run.smithi073.stdout:4753 read 0x575e494 thru 0x57642fb (0x5e68 bytes) 2024-09-21T14:22:14.179 INFO:teuthology.orchestra.run.smithi073.stdout:4754 read 0x8724b96 thru 0x8726231 (0x169c bytes) 2024-09-21T14:22:14.180 INFO:teuthology.orchestra.run.smithi073.stdout:4755 punch from 0x918879d to 0x918a15f, (0x19c2 bytes) 2024-09-21T14:22:14.180 INFO:teuthology.orchestra.run.smithi073.stdout:4756 write 0x71329e thru 0x72319a (0xfefd bytes) 2024-09-21T14:22:14.181 INFO:teuthology.orchestra.run.smithi073.stdout:4757 punch from 0x157d630 to 0x1586d57, (0x9727 bytes) 2024-09-21T14:22:14.181 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.182 INFO:teuthology.orchestra.run.smithi073.stdout:4759 trunc from 0xb2e4762 to 0xa3b74a7 2024-09-21T14:22:14.194 INFO:teuthology.orchestra.run.smithi073.stdout:4760 trunc from 0xa3b74a7 to 0x5c9b66c 2024-09-21T14:22:14.198 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:14.198 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:14.205 INFO:teuthology.orchestra.run.smithi073.stdout:4761 read 0x2d55b1b thru 0x2d57751 (0x1c37 bytes) 2024-09-21T14:22:14.206 INFO:teuthology.orchestra.run.smithi073.stdout:4762 writesame 0x5344af thru 0x541ed3 data_size 0x33(0xda25 bytes) 2024-09-21T14:22:14.208 INFO:teuthology.orchestra.run.smithi073.stdout:4764 write 0x22bcd50 thru 0x22c7576 (0xa827 bytes) 2024-09-21T14:22:14.211 INFO:teuthology.orchestra.run.smithi073.stdout:4765 read 0x143fe02 thru 0x144ed5c (0xef5b bytes) 2024-09-21T14:22:14.216 INFO:teuthology.orchestra.run.smithi073.stdout:4766 trunc from 0x5c9b66c to 0x98d14fe 2024-09-21T14:22:14.216 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:14.221 INFO:teuthology.orchestra.run.smithi073.stdout:4767 write 0x8b84dbd thru 0x8b89810 (0x4a54 bytes) 2024-09-21T14:22:14.223 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:14.223+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:14.224 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.224 INFO:teuthology.orchestra.run.smithi073.stdout:4770 write 0x1eeb283 thru 0x1ef022b (0x4fa9 bytes) 2024-09-21T14:22:14.226 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.226 INFO:teuthology.orchestra.run.smithi073.stdout:4772 write 0x1d09a00 thru 0x1d12d68 (0x9369 bytes) 2024-09-21T14:22:14.227 INFO:teuthology.orchestra.run.smithi073.stdout:4775 write 0x84679e0 thru 0x846e414 (0x6a35 bytes) 2024-09-21T14:22:14.233 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:14.233 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:14.238 INFO:teuthology.orchestra.run.smithi073.stdout:4776 write 0xa9fb86e thru 0xaa01848 (0x5fdb bytes) 2024-09-21T14:22:14.239 INFO:teuthology.orchestra.run.smithi073.stdout:4777 writesame 0xa89a050 thru 0xa8a65c8 data_size 0x9(0xc579 bytes) 2024-09-21T14:22:14.251 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:14.251 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:14.252 INFO:teuthology.orchestra.run.smithi073.stdout:4778 write 0xd44ae5e thru 0xd455045 (0xa1e8 bytes) 2024-09-21T14:22:14.256 INFO:teuthology.orchestra.run.smithi073.stdout:4780 write 0x85449a4 thru 0x854b3f2 (0x6a4f bytes) 2024-09-21T14:22:14.260 INFO:teuthology.orchestra.run.smithi073.stdout:4781 punch from 0x20fd883 to 0x2106efb, (0x9678 bytes) 2024-09-21T14:22:14.260 INFO:teuthology.orchestra.run.smithi073.stdout:4783 write 0x7ee09ac thru 0x7ee4e00 (0x4455 bytes) 2024-09-21T14:22:14.276 INFO:teuthology.orchestra.run.smithi073.stdout:4785 trunc from 0xd455046 to 0x17c2c2b 2024-09-21T14:22:14.276 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:14.276 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:14.287 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:14.292 INFO:teuthology.orchestra.run.smithi073.stdout:4787 read 0x781d93 thru 0x790f9a (0xf208 bytes) 2024-09-21T14:22:14.293 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.293 INFO:teuthology.orchestra.run.smithi073.stdout:4790 read 0x1541617 thru 0x154db3b (0xc525 bytes) 2024-09-21T14:22:14.305 INFO:teuthology.orchestra.run.smithi073.stdout:4791 writesame 0x6f44d12 thru 0x6f517b8 data_size 0x1(0xcaa7 bytes) 2024-09-21T14:22:14.305 INFO:teuthology.orchestra.run.smithi073.stdout:4792 write 0x5686413 thru 0x5686ba2 (0x790 bytes) 2024-09-21T14:22:14.307 INFO:teuthology.orchestra.run.smithi073.stdout:4793 writesame 0x495b5f4 thru 0x495d9ee data_size 0x1(0x23fb bytes) 2024-09-21T14:22:14.313 INFO:teuthology.orchestra.run.smithi073.stdout:4794 read 0x194fb0 thru 0x1a46a9 (0xf6fa bytes) 2024-09-21T14:22:14.313 INFO:teuthology.orchestra.run.smithi073.stdout:4796 punch from 0x1a5d6a6 to 0x1a5d94d, (0x2a7 bytes) 2024-09-21T14:22:14.314 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.314 INFO:teuthology.orchestra.run.smithi073.stdout:4800 write 0x55a013c thru 0x55ae639 (0xe4fe bytes) 2024-09-21T14:22:14.324 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:14.324+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:14.329 INFO:teuthology.orchestra.run.smithi073.stdout:4802 write 0xd5679c4 thru 0xd569065 (0x16a2 bytes) 2024-09-21T14:22:14.332 INFO:teuthology.orchestra.run.smithi073.stdout:4803 writesame 0x27fa028 thru 0x27fbabc data_size 0x1(0x1a95 bytes) 2024-09-21T14:22:14.335 INFO:teuthology.orchestra.run.smithi073.stdout:4804 write 0x20db63f thru 0x20e6c9d (0xb65f bytes) 2024-09-21T14:22:14.337 INFO:teuthology.orchestra.run.smithi073.stdout:4806 writesame 0x39f8c82 thru 0x39ff6cb data_size 0xf(0x6a4a bytes) 2024-09-21T14:22:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:4808 read 0x6820090 thru 0x682e161 (0xe0d2 bytes) 2024-09-21T14:22:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:4809 read 0x92d0990 thru 0x92e07ef (0xfe60 bytes) 2024-09-21T14:22:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:4810 read 0x1387b00 thru 0x138ae8a (0x338b bytes) 2024-09-21T14:22:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:4811 punch from 0x39bb6bc to 0x39bf09b, (0x39df bytes) 2024-09-21T14:22:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:4812 write 0x6e34c0c thru 0x6e448fe (0xfcf3 bytes) 2024-09-21T14:22:14.344 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.344 INFO:teuthology.orchestra.run.smithi073.stdout:4815 read 0x3ba1319 thru 0x3bacca0 (0xb988 bytes) 2024-09-21T14:22:14.344 INFO:teuthology.orchestra.run.smithi073.stdout:4816 read 0x193b2f5 thru 0x19418db (0x65e7 bytes) 2024-09-21T14:22:14.344 INFO:teuthology.orchestra.run.smithi073.stdout:4817 read 0x363f7ff thru 0x3642b8d (0x338f bytes) 2024-09-21T14:22:14.345 INFO:teuthology.orchestra.run.smithi073.stdout:4818 write 0x511f6ca thru 0x5122a72 (0x33a9 bytes) 2024-09-21T14:22:14.347 INFO:teuthology.orchestra.run.smithi073.stdout:4819 writesame 0x7a6afa2 thru 0x7a7598b data_size 0x1a(0xa9ea bytes) 2024-09-21T14:22:14.351 INFO:teuthology.orchestra.run.smithi073.stdout:4823 read 0xd128a6d thru 0xd1382eb (0xf87f bytes) 2024-09-21T14:22:14.351 INFO:teuthology.orchestra.run.smithi073.stdout:4824 writesame 0xabff1a5 thru 0xac0630d data_size 0x1(0x7169 bytes) 2024-09-21T14:22:14.369 INFO:teuthology.orchestra.run.smithi073.stdout:4825 writesame 0x4ddd781 thru 0x4deb9de data_size 0x5f(0xe25e bytes) 2024-09-21T14:22:14.375 INFO:teuthology.orchestra.run.smithi073.stdout:4826 read 0x174f01c thru 0x1759409 (0xa3ee bytes) 2024-09-21T14:22:14.379 INFO:teuthology.orchestra.run.smithi073.stdout:4827 trunc from 0xd569066 to 0xa93b8e3 2024-09-21T14:22:14.389 INFO:teuthology.orchestra.run.smithi073.stdout:4828 read 0x9a7d8bf thru 0x9a82632 (0x4d74 bytes) 2024-09-21T14:22:14.389 INFO:teuthology.orchestra.run.smithi073.stdout:4829 read 0xa15d560 thru 0xa162a38 (0x54d9 bytes) 2024-09-21T14:22:14.389 INFO:teuthology.orchestra.run.smithi073.stdout:4831 read 0x7791f14 thru 0x77a0842 (0xe92f bytes) 2024-09-21T14:22:14.390 INFO:teuthology.orchestra.run.smithi073.stdout:4832 write 0x8b2793 thru 0x8b7ea7 (0x5715 bytes) 2024-09-21T14:22:14.391 INFO:teuthology.orchestra.run.smithi073.stdout:4833 punch from 0x57bff7 to 0x589737, (0xd740 bytes) 2024-09-21T14:22:14.398 INFO:teuthology.orchestra.run.smithi073.stdout:4835 writesame 0xcaa6c05 thru 0xcaaaa16 data_size 0x23(0x3e12 bytes) 2024-09-21T14:22:14.398 INFO:teuthology.orchestra.run.smithi073.stdout:4837 read 0x23af38c thru 0x23b202f (0x2ca4 bytes) 2024-09-21T14:22:14.399 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:14.399 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:14.399 INFO:teuthology.orchestra.run.smithi073.stdout:4838 punch from 0x660a8be to 0x660b750, (0xe92 bytes) 2024-09-21T14:22:14.399 INFO:teuthology.orchestra.run.smithi073.stdout:4839 read 0x4d17bd8 thru 0x4d214d1 (0x98fa bytes) 2024-09-21T14:22:14.400 INFO:teuthology.orchestra.run.smithi073.stdout:4840 read 0x1988759 thru 0x198d404 (0x4cac bytes) 2024-09-21T14:22:14.400 INFO:teuthology.orchestra.run.smithi073.stdout:4841 writesame 0xbcc01f7 thru 0xbcc8c54 data_size 0x59(0x8a5e bytes) 2024-09-21T14:22:14.408 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:14.408 INFO:teuthology.orchestra.run.smithi073.stdout:4842 write 0xd9f2960 thru 0xd9f6627 (0x3cc8 bytes) 2024-09-21T14:22:14.415 INFO:teuthology.orchestra.run.smithi073.stdout:4843 trunc from 0xd9f6628 to 0x9ddd5c5 2024-09-21T14:22:14.425 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:14.425+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:14.428 INFO:teuthology.orchestra.run.smithi073.stdout:4844 read 0x8ebf61f thru 0x8ec2826 (0x3208 bytes) 2024-09-21T14:22:14.428 INFO:teuthology.orchestra.run.smithi073.stdout:4845 punch from 0x577696 to 0x579209, (0x1b73 bytes) 2024-09-21T14:22:14.428 INFO:teuthology.orchestra.run.smithi073.stdout:4846 trunc from 0x9ddd5c5 to 0xa23818e 2024-09-21T14:22:14.432 INFO:teuthology.orchestra.run.smithi073.stdout:4848 write 0x3bb2ae0 thru 0x3bb3d4b (0x126c bytes) 2024-09-21T14:22:14.434 INFO:teuthology.orchestra.run.smithi073.stdout:4849 read 0x6acf469 thru 0x6adefae (0xfb46 bytes) 2024-09-21T14:22:14.439 INFO:teuthology.orchestra.run.smithi073.stdout:4850 write 0xa2f6ddf thru 0xa303cfe (0xcf20 bytes) 2024-09-21T14:22:14.448 INFO:teuthology.orchestra.run.smithi073.stdout:4851 writesame 0xb30c38b thru 0xb30e7df data_size 0x1(0x2455 bytes) 2024-09-21T14:22:14.455 INFO:teuthology.orchestra.run.smithi073.stdout:4852 trunc from 0xb30e7e0 to 0x4fe0c96 2024-09-21T14:22:14.470 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:14.472 INFO:teuthology.orchestra.run.smithi073.stdout:4853 writesame 0x3d5fc34 thru 0x3d6dacf data_size 0x12(0xde9c bytes) 2024-09-21T14:22:14.482 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:22:14.490 INFO:teuthology.orchestra.run.smithi073.stdout:4854 writesame 0x7c9305d thru 0x7ca22ab data_size 0x57(0xf24f bytes) 2024-09-21T14:22:14.495 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:14.495 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:14.503 INFO:teuthology.orchestra.run.smithi073.stdout:4855 write 0xe3a828c thru 0xe3aee17 (0x6b8c bytes) 2024-09-21T14:22:14.506 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.512 INFO:teuthology.orchestra.run.smithi073.stdout:4858 trunc from 0xe3aee18 to 0x9c52177 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10529 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808552614999996, 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032822136000000002, 2024-09-21T14:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10529 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709885+0000", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709884+0000", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709886+0000", 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1790000000000001e-06 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709889+0000", 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.932e-06 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709892+0000", 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742551+0000", 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032658622999999998 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742637+0000", 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6587000000000003e-05 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742707+0000", 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9580999999999997e-05 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10532 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808541304999999, 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033009504000000002, 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10532 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709896+0000", 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709897+0000", 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8220000000000001e-06 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709903+0000", 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709908+0000", 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5589999999999999e-06 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742749+0000", 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032841018 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742835+0000", 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6388e-05 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742905+0000", 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0474000000000003e-05 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10546 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.528 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808525928999998, 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033137591000000001, 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10546 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709911+0000", 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709913+0000", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.697e-06 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709920+0000", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2779999999999996e-06 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709925+0000", 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8189999999999999e-06 2024-09-21T14:22:14.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742936+0000", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033011556999999997 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.742993+0000", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7089999999999997e-05 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743049+0000", 2024-09-21T14:22:14.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5445999999999997e-05 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10548 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808508400999997, 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033320561999999998, 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10548 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709929+0000", 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709934+0000", 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709942+0000", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7680000000000008e-06 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709945+0000", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9809999999999999e-06 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743084+0000", 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033138728999999999 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743163+0000", 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8781000000000003e-05 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743249+0000", 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6907e-05 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808489450000003, 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808473069999998, 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.542 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:14.546 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808463842000002, 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:22:14.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:22:14.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808452445999997, 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.551 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:22:14.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.555 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808429719000003, 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:22:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:22:14.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808413463000001, 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:14.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:22:14.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.563 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808398707999999, 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:22:14.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.568 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808389654000003, 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:22:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:22:14.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.573 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808378609999998, 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:22:14.577 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808363208999999, 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808346511000003, 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:22:14.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.586 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808330904000002, 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:22:14.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.590 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808313781999999, 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.591 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:22:14.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808295170999997, 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:22:14.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:14.599 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808275840999997, 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.808256204000003, 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:14.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:14.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:22:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:14.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:14.608 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:14.609 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:14.609 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:14.611 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:14.611 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:14.608+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:14.611 INFO:teuthology.orchestra.run.smithi073.stdout:4860 write 0x2f8b941 thru 0x2f8f70d (0x3dcd bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4861 trunc from 0x9c52177 to 0xd57091c 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4862 punch from 0x78a0982 to 0x78a23d1, (0x1a4f bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4863 write 0x3532c37 thru 0x3535ca0 (0x306a bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4864 punch from 0xc2a654b to 0xc2a9fa9, (0x3a5e bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4865 read 0x7fcd2b4 thru 0x7fd8610 (0xb35d bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4866 write 0x74e8bbe thru 0x74f4c8e (0xc0d1 bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:4867 writesame 0xbd1a18c thru 0xbd1f98a data_size 0x3(0x57ff bytes) 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:14.613 INFO:teuthology.orchestra.run.smithi073.stdout:4869 read 0x1a19b61 thru 0x1a22386 (0x8826 bytes) 2024-09-21T14:22:14.614 INFO:teuthology.orchestra.run.smithi073.stdout:4870 trunc from 0xd57091c to 0x6d25c1c 2024-09-21T14:22:14.614 INFO:teuthology.orchestra.run.smithi073.stdout:4871 write 0x9e33703 thru 0x9e34663 (0xf61 bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4872 read 0x53462da thru 0x534ff1e (0x9c45 bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4873 punch from 0x3cee215 to 0x3cf2b60, (0x494b bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4876 write 0xbcafb47 thru 0xbcb44b5 (0x496f bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4878 read 0x52d4a54 thru 0x52dafe2 (0x658f bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4881 writesame 0x750f201 thru 0x751262c data_size 0x2a(0x342c bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4883 write 0x11f3523 thru 0x11f6f39 (0x3a17 bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4886 write 0x59b542a thru 0x59b72e8 (0x1ebf bytes) 2024-09-21T14:22:14.615 INFO:teuthology.orchestra.run.smithi073.stdout:4888 write 0xe06a2e9 thru 0xe06a65e (0x376 bytes) 2024-09-21T14:22:14.621 INFO:teuthology.orchestra.run.smithi073.stdout:4890 trunc from 0xe06a65f to 0x1502e4 2024-09-21T14:22:14.641 INFO:teuthology.orchestra.run.smithi073.stdout:4891 read 0x14f97e thru 0x1502e3 (0x966 bytes) 2024-09-21T14:22:14.663 INFO:teuthology.orchestra.run.smithi073.stdout:4894 write 0xd623173 thru 0xd629690 (0x651e bytes) 2024-09-21T14:22:14.666 INFO:teuthology.orchestra.run.smithi073.stdout:4895 writesame 0x4266af6 thru 0x42765df data_size 0x1(0xfaea bytes) 2024-09-21T14:22:14.692 INFO:teuthology.orchestra.run.smithi073.stdout:4896 trunc from 0xd629691 to 0x66062a2 2024-09-21T14:22:14.704 INFO:teuthology.orchestra.run.smithi073.stdout:4897 read 0x36f50ca thru 0x36fb08a (0x5fc1 bytes) 2024-09-21T14:22:14.704 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.707 INFO:teuthology.orchestra.run.smithi073.stdout:4899 trunc from 0x66062a2 to 0x95da4a4 2024-09-21T14:22:14.709 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:14.710+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:14.712 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.712 INFO:teuthology.orchestra.run.smithi073.stdout:4901 write 0x8665996 thru 0x8666e98 (0x1503 bytes) 2024-09-21T14:22:14.715 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.715 INFO:teuthology.orchestra.run.smithi073.stdout:4903 write 0x255dcea thru 0x255ead0 (0xde7 bytes) 2024-09-21T14:22:14.719 INFO:teuthology.orchestra.run.smithi073.stdout:4905 trunc from 0x95da4a4 to 0xc256418 2024-09-21T14:22:14.724 INFO:teuthology.orchestra.run.smithi073.stdout:4906 read 0x1fe5422 thru 0x1fe8520 (0x30ff bytes) 2024-09-21T14:22:14.725 INFO:teuthology.orchestra.run.smithi073.stdout:4910 punch from 0xadcb95d to 0xadce8e1, (0x2f84 bytes) 2024-09-21T14:22:14.732 INFO:teuthology.orchestra.run.smithi073.stdout:4911 write 0xdfe3945 thru 0xdff3024 (0xf6e0 bytes) 2024-09-21T14:22:14.736 INFO:teuthology.orchestra.run.smithi073.stdout:4915 write 0x63079c5 thru 0x630f11a (0x7756 bytes) 2024-09-21T14:22:14.739 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.739 INFO:teuthology.orchestra.run.smithi073.stdout:4918 writesame 0xce3d6d4 thru 0xce4a795 data_size 0x9(0xd0c2 bytes) 2024-09-21T14:22:14.744 INFO:teuthology.orchestra.run.smithi073.stdout:4920 writesame 0x7922e07 thru 0x7926b68 data_size 0x36(0x3d62 bytes) 2024-09-21T14:22:14.747 INFO:teuthology.orchestra.run.smithi073.stdout:4921 write 0xdf0a0f5 thru 0xdf0f186 (0x5092 bytes) 2024-09-21T14:22:14.748 INFO:teuthology.orchestra.run.smithi073.stdout:4922 read 0x3116462 thru 0x311cd54 (0x68f3 bytes) 2024-09-21T14:22:14.749 INFO:teuthology.orchestra.run.smithi073.stdout:4923 write 0x459afb0 thru 0x45a1703 (0x6754 bytes) 2024-09-21T14:22:14.752 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.752 INFO:teuthology.orchestra.run.smithi073.stdout:4925 write 0x45a7b85 thru 0x45b0f22 (0x939e bytes) 2024-09-21T14:22:14.755 INFO:teuthology.orchestra.run.smithi073.stdout:4926 writesame 0x9691fa4 thru 0x96a1745 data_size 0x1d(0xf7a2 bytes) 2024-09-21T14:22:14.762 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:14.762 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:14.771 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:14.774 INFO:teuthology.orchestra.run.smithi073.stdout:4927 trunc from 0xdff3025 to 0x174e623 2024-09-21T14:22:14.788 INFO:teuthology.orchestra.run.smithi073.stdout:4928 read 0xd8d05b thru 0xd9618c (0x9132 bytes) 2024-09-21T14:22:14.802 INFO:teuthology.orchestra.run.smithi073.stdout:4929 write 0x848775e thru 0x8494c04 (0xd4a7 bytes) 2024-09-21T14:22:14.810 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:14.810+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:14.813 INFO:teuthology.orchestra.run.smithi073.stdout:4930 writesame 0xa281663 thru 0xa28deaf data_size 0x1(0xc84d bytes) 2024-09-21T14:22:14.814 INFO:teuthology.orchestra.run.smithi073.stdout:4931 write 0x7cde8f6 thru 0x7ce51b4 (0x68bf bytes) 2024-09-21T14:22:14.824 INFO:teuthology.orchestra.run.smithi073.stdout:4932 write 0xbcd0928 thru 0xbcd0c49 (0x322 bytes) 2024-09-21T14:22:14.827 INFO:teuthology.orchestra.run.smithi073.stdout:4933 write 0x1b169e1 thru 0x1b2060c (0x9c2c bytes) 2024-09-21T14:22:14.829 INFO:teuthology.orchestra.run.smithi073.stdout:4934 writesame 0x15adb5a thru 0x15b907c data_size 0x29(0xb523 bytes) 2024-09-21T14:22:14.831 INFO:teuthology.orchestra.run.smithi073.stdout:4935 write 0x1314652 thru 0x131f1c6 (0xab75 bytes) 2024-09-21T14:22:14.832 INFO:teuthology.orchestra.run.smithi073.stdout:4936 write 0x2f1d75a thru 0x2f29f26 (0xc7cd bytes) 2024-09-21T14:22:14.834 INFO:teuthology.orchestra.run.smithi073.stdout:4937 read 0x811d99a thru 0x811f7ec (0x1e53 bytes) 2024-09-21T14:22:14.834 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.834 INFO:teuthology.orchestra.run.smithi073.stdout:4939 writesame 0x2173631 thru 0x217baa6 data_size 0x5(0x8476 bytes) 2024-09-21T14:22:14.837 INFO:teuthology.orchestra.run.smithi073.stdout:4940 write 0x9f31665 thru 0x9f3bc13 (0xa5af bytes) 2024-09-21T14:22:14.841 INFO:teuthology.orchestra.run.smithi073.stdout:4941 write 0x7d78be2 thru 0x7d885a1 (0xf9c0 bytes) 2024-09-21T14:22:14.843 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.844 INFO:teuthology.orchestra.run.smithi073.stdout:4943 read 0xf65caa thru 0xf66052 (0x3a9 bytes) 2024-09-21T14:22:14.846 INFO:teuthology.orchestra.run.smithi073.stdout:4944 trunc from 0xbcd0c4a to 0xa1f1cf5 2024-09-21T14:22:14.858 INFO:teuthology.orchestra.run.smithi073.stdout:4945 read 0x63823c6 thru 0x6387868 (0x54a3 bytes) 2024-09-21T14:22:14.858 INFO:teuthology.orchestra.run.smithi073.stdout:4946 read 0x8040a7a thru 0x804ebb1 (0xe138 bytes) 2024-09-21T14:22:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:4947 trunc from 0xa1f1cf5 to 0xb17071 2024-09-21T14:22:14.884 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:14.884 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:14.884 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:14.884 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:14.884 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:14.885 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.885 INFO:teuthology.orchestra.run.smithi073.stdout:4954 read 0x91103f thru 0x916fce (0x5f90 bytes) 2024-09-21T14:22:14.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:14.898 INFO:teuthology.orchestra.run.smithi073.stdout:4955 trunc from 0xb17071 to 0xbd6e70f 2024-09-21T14:22:14.904 INFO:teuthology.orchestra.run.smithi073.stdout:4957 punch from 0xba0a07f to 0xba0b92e, (0x18af bytes) 2024-09-21T14:22:14.904 INFO:teuthology.orchestra.run.smithi073.stdout:4958 punch from 0x68f5063 to 0x68fe3f3, (0x9390 bytes) 2024-09-21T14:22:14.904 INFO:teuthology.orchestra.run.smithi073.stdout:4959 write 0x21155ed thru 0x211b8a8 (0x62bc bytes) 2024-09-21T14:22:14.912 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:14.912+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:14.917 INFO:teuthology.orchestra.run.smithi073.stdout:4960 trunc from 0xbd6e70f to 0x21c9718 2024-09-21T14:22:14.928 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:14.928 INFO:teuthology.orchestra.run.smithi073.stdout:4963 read 0x1799bdf thru 0x17a7a7f (0xdea1 bytes) 2024-09-21T14:22:14.928 INFO:teuthology.orchestra.run.smithi073.stdout:4965 read 0x1585864 thru 0x1590e50 (0xb5ed bytes) 2024-09-21T14:22:14.946 INFO:teuthology.orchestra.run.smithi073.stdout:4967 write 0xc32c8c6 thru 0xc32f538 (0x2c73 bytes) 2024-09-21T14:22:14.949 INFO:teuthology.orchestra.run.smithi073.stdout:4969 writesame 0xb296030 thru 0xb2a4ab8 data_size 0x1(0xea89 bytes) 2024-09-21T14:22:14.981 INFO:teuthology.orchestra.run.smithi073.stdout:4970 trunc from 0xc32f539 to 0x50c77a1 2024-09-21T14:22:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571504038999997, 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.004 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571487396999999, 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:15.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571462347000001, 2024-09-21T14:22:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:15.016 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571444194999998, 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.020 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571420957999997, 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:15.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:15.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571399890999999, 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.025 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571383531000002, 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571353250999998, 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571338113000003, 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571318517000002, 2024-09-21T14:22:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571309739999997, 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571297379000001, 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571287962, 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571266770999998, 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571249663000003, 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571232719999998, 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571216341000003, 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571174980999999, 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.571155300000001, 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.166266853, 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.093 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:15.096 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:15.096 INFO:teuthology.orchestra.run.smithi073.stdout:4971 write 0xe758ffb thru 0xe75975e (0x764 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4972 trunc from 0xe75975f to 0x4174e7 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4974 writesame 0xda64575 thru 0xda6a6c3 data_size 0x1(0x614f bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4976 punch from 0x5beace7 to 0x5bf3bfc, (0x8f15 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4977 write 0xd8f0365 thru 0xd8f71db (0x6e77 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4978 punch from 0x2619eb to 0x26ba65, (0xa07a bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4979 write 0xdb3a19a thru 0xdb459a4 (0xb80b bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4981 read 0x8d1e71 thru 0x8db0d4 (0x9264 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4982 read 0xa972430 thru 0xa97b121 (0x8cf2 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4983 trunc from 0xdb459a5 to 0x8ef3542 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4985 write 0x17c7f71 thru 0x17d20b2 (0xa142 bytes) 2024-09-21T14:22:15.097 INFO:teuthology.orchestra.run.smithi073.stdout:4987 write 0x8d4164f thru 0x8d478d6 (0x6288 bytes) 2024-09-21T14:22:15.098 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:15.095+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:15.098 INFO:teuthology.orchestra.run.smithi073.stdout:4988 read 0x60fb05e thru 0x6109f3f (0xeee2 bytes) 2024-09-21T14:22:15.108 INFO:teuthology.orchestra.run.smithi073.stdout:4989 writesame 0xe6efd2a thru 0xe6fe514 data_size 0x1(0xe7eb bytes) 2024-09-21T14:22:15.108 INFO:teuthology.orchestra.run.smithi073.stdout:4990 read 0x82900d9 thru 0x829a962 (0xa88a bytes) 2024-09-21T14:22:15.108 INFO:teuthology.orchestra.run.smithi073.stdout:4991 read 0x14b9e41 thru 0x14c997f (0xfb3f bytes) 2024-09-21T14:22:15.117 INFO:teuthology.orchestra.run.smithi073.stdout:4992 trunc from 0xe6fe515 to 0x735cf5b 2024-09-21T14:22:15.126 INFO:teuthology.orchestra.run.smithi073.stdout:4993 read 0x287831a thru 0x287c79a (0x4481 bytes) 2024-09-21T14:22:15.131 INFO:teuthology.orchestra.run.smithi073.stdout:4994 trunc from 0x735cf5b to 0x31708b6 2024-09-21T14:22:15.139 INFO:teuthology.orchestra.run.smithi073.stdout:4995 read 0x1e70eef thru 0x1e76558 (0x566a bytes) 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:4996 punch from 0x21b6019 to 0x21bfb18, (0x9aff bytes) 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:4997 punch from 0x1326eb to 0x1359c8, (0x32dd bytes) 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:4998 read 0x1ce7678 thru 0x1cf40da (0xca63 bytes) 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.140 INFO:teuthology.orchestra.run.smithi073.stdout:5002 punch from 0x22696b to 0x2341e1, (0xd876 bytes) 2024-09-21T14:22:15.155 INFO:teuthology.orchestra.run.smithi073.stdout:5003 writesame 0xb06a121 thru 0xb079cd6 data_size 0xb(0xfbb6 bytes) 2024-09-21T14:22:15.155 INFO:teuthology.orchestra.run.smithi073.stdout:5006 read 0x36f4656 thru 0x3703621 (0xefcc bytes) 2024-09-21T14:22:15.161 INFO:teuthology.orchestra.run.smithi073.stdout:5008 write 0xc4c4967 thru 0xc4cbed2 (0x756c bytes) 2024-09-21T14:22:15.164 INFO:teuthology.orchestra.run.smithi073.stdout:5009 read 0x303718e thru 0x303d7f2 (0x6665 bytes) 2024-09-21T14:22:15.165 INFO:teuthology.orchestra.run.smithi073.stdout:5012 trunc from 0xc4cbed3 to 0xcc2a009 2024-09-21T14:22:15.170 INFO:teuthology.orchestra.run.smithi073.stdout:5013 read 0x1805391 thru 0x1811a62 (0xc6d2 bytes) 2024-09-21T14:22:15.170 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.170 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.170 INFO:teuthology.orchestra.run.smithi073.stdout:5017 read 0x61fecf8 thru 0x6208515 (0x981e bytes) 2024-09-21T14:22:15.177 INFO:teuthology.orchestra.run.smithi073.stdout:5019 writesame 0xdfeb275 thru 0xdff2639 data_size 0x25(0x73c5 bytes) 2024-09-21T14:22:15.178 INFO:teuthology.orchestra.run.smithi073.stdout:5021 write 0x91d98e4 thru 0x91e5cf0 (0xc40d bytes) 2024-09-21T14:22:15.182 INFO:teuthology.orchestra.run.smithi073.stdout:5022 punch from 0x36ee871 to 0x36f40b9, (0x5848 bytes) 2024-09-21T14:22:15.193 INFO:teuthology.orchestra.run.smithi073.stdout:5023 trunc from 0xdff263a to 0x4926477 2024-09-21T14:22:15.196 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:15.195+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:15.202 INFO:teuthology.orchestra.run.smithi073.stdout:5025 read 0x6fa4c5 thru 0x6ff251 (0x4d8d bytes) 2024-09-21T14:22:15.203 INFO:teuthology.orchestra.run.smithi073.stdout:5026 read 0x1f8319b thru 0x1f8fa64 (0xc8ca bytes) 2024-09-21T14:22:15.203 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.203 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi073.stdout:5033 punch from 0xcd780e to 0xce2c69, (0xb45b bytes) 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi073.stdout:5034 punch from 0x1f9e38f to 0x1faa21e, (0xbe8f bytes) 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi073.stdout:5035 read 0x11f7651 thru 0x11fa6bf (0x306f bytes) 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:15.204 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:15.213 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:15.220 INFO:teuthology.orchestra.run.smithi073.stdout:5036 write 0xdbedf1c thru 0xdbf2b38 (0x4c1d bytes) 2024-09-21T14:22:15.224 INFO:teuthology.orchestra.run.smithi073.stdout:5039 punch from 0x948fcc7 to 0x9490c93, (0xfcc bytes) 2024-09-21T14:22:15.224 INFO:teuthology.orchestra.run.smithi073.stdout:5040 writesame 0xce3503c thru 0xce3923a data_size 0x1f(0x41ff bytes) 2024-09-21T14:22:15.226 INFO:teuthology.orchestra.run.smithi073.stdout:5041 punch from 0xab498e0 to 0xab4b7a6, (0x1ec6 bytes) 2024-09-21T14:22:15.226 INFO:teuthology.orchestra.run.smithi073.stdout:5043 punch from 0x78c02ca to 0x78cc77b, (0xc4b1 bytes) 2024-09-21T14:22:15.226 INFO:teuthology.orchestra.run.smithi073.stdout:5044 write 0x26c2a10 thru 0x26c731a (0x490b bytes) 2024-09-21T14:22:15.228 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.236 INFO:teuthology.orchestra.run.smithi073.stdout:5046 trunc from 0xdbf2b39 to 0x7057de4 2024-09-21T14:22:15.254 INFO:teuthology.orchestra.run.smithi073.stdout:5047 write 0xa45ad99 thru 0xa460904 (0x5b6c bytes) 2024-09-21T14:22:15.258 INFO:teuthology.orchestra.run.smithi073.stdout:5048 punch from 0x1c6117 to 0x1d5086, (0xef6f bytes) 2024-09-21T14:22:15.264 INFO:teuthology.orchestra.run.smithi073.stdout:5049 write 0xbbc5dbd thru 0xbbcf877 (0x9abb bytes) 2024-09-21T14:22:15.275 INFO:teuthology.orchestra.run.smithi073.stdout:5050 writesame 0xebbf1ec thru 0xebc06cb data_size 0x8(0x14e0 bytes) 2024-09-21T14:22:15.290 INFO:teuthology.orchestra.run.smithi073.stdout:5051 trunc from 0xebc06cc to 0x265d0e4 2024-09-21T14:22:15.299 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:15.299+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:15.325 INFO:teuthology.orchestra.run.smithi073.stdout:5052 write 0xea5b962 thru 0xea651fb (0x989a bytes) 2024-09-21T14:22:15.330 INFO:teuthology.orchestra.run.smithi073.stdout:5055 write 0x4a01269 thru 0x4a0cf49 (0xbce1 bytes) 2024-09-21T14:22:15.333 INFO:teuthology.orchestra.run.smithi073.stdout:5056 write 0x20336c6 thru 0x2033c09 (0x544 bytes) 2024-09-21T14:22:15.334 INFO:teuthology.orchestra.run.smithi073.stdout:5058 write 0x71c903f thru 0x71d774f (0xe711 bytes) 2024-09-21T14:22:15.340 INFO:teuthology.orchestra.run.smithi073.stdout:5059 writesame 0x9531e thru 0x9c355 data_size 0x54(0x7038 bytes) 2024-09-21T14:22:15.342 INFO:teuthology.orchestra.run.smithi073.stdout:5060 read 0xca27b47 thru 0xca2dd91 (0x624b bytes) 2024-09-21T14:22:15.342 INFO:teuthology.orchestra.run.smithi073.stdout:5061 writesame 0x90d15df thru 0x90d764f data_size 0x1(0x6071 bytes) 2024-09-21T14:22:15.356 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.356 INFO:teuthology.orchestra.run.smithi073.stdout:5063 write 0x1b27cfb thru 0x1b31088 (0x938e bytes) 2024-09-21T14:22:15.358 INFO:teuthology.orchestra.run.smithi073.stdout:5064 punch from 0x2bc8d09 to 0x2bd5de0, (0xd0d7 bytes) 2024-09-21T14:22:15.358 INFO:teuthology.orchestra.run.smithi073.stdout:5065 write 0x50514d2 thru 0x505ab83 (0x96b2 bytes) 2024-09-21T14:22:15.363 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:15.363 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:15.363 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:15.363 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:15.363 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:15.371 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:15.372 INFO:teuthology.orchestra.run.smithi073.stdout:5067 trunc from 0xea651fc to 0x55e1c9d 2024-09-21T14:22:15.385 INFO:teuthology.orchestra.run.smithi073.stdout:5068 read 0x1dcc943 thru 0x1dcfd99 (0x3457 bytes) 2024-09-21T14:22:15.385 INFO:teuthology.orchestra.run.smithi073.stdout:5069 punch from 0x3fce164 to 0x3fdd7fd, (0xf699 bytes) 2024-09-21T14:22:15.395 INFO:teuthology.orchestra.run.smithi073.stdout:5072 write 0x8a58092 thru 0x8a5dd8d (0x5cfc bytes) 2024-09-21T14:22:15.397 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.398 INFO:teuthology.orchestra.run.smithi073.stdout:5074 read 0x4662d12 thru 0x466be97 (0x9186 bytes) 2024-09-21T14:22:15.398 INFO:teuthology.orchestra.run.smithi073.stdout:5076 trunc from 0x8a5dd8e to 0x917709a 2024-09-21T14:22:15.403 INFO:teuthology.orchestra.run.smithi073.stdout:5077 read 0x4b6c292 thru 0x4b6e25a (0x1fc9 bytes) 2024-09-21T14:22:15.403 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.404 INFO:teuthology.orchestra.run.smithi073.stdout:5081 read 0x68251a3 thru 0x68335a0 (0xe3fe bytes) 2024-09-21T14:22:15.404 INFO:teuthology.orchestra.run.smithi073.stdout:5082 read 0x30f4134 thru 0x30f5752 (0x161f bytes) 2024-09-21T14:22:15.404 INFO:teuthology.orchestra.run.smithi073.stdout:5083 punch from 0x9073b94 to 0x907e2db, (0xa747 bytes) 2024-09-21T14:22:15.414 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:15.413+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:15.416 INFO:teuthology.orchestra.run.smithi073.stdout:5084 write 0xd69f257 thru 0xd6ad91c (0xe6c6 bytes) 2024-09-21T14:22:15.426 INFO:teuthology.orchestra.run.smithi073.stdout:5085 write 0xd94b020 thru 0xd957380 (0xc361 bytes) 2024-09-21T14:22:15.434 INFO:teuthology.orchestra.run.smithi073.stdout:5086 trunc from 0xd957381 to 0xcf5f01a 2024-09-21T14:22:15.442 INFO:teuthology.orchestra.run.smithi073.stdout:5087 read 0xa7a55ca thru 0xa7a6f87 (0x19be bytes) 2024-09-21T14:22:15.442 INFO:teuthology.orchestra.run.smithi073.stdout:5088 writesame 0x60d91dc thru 0x60e4d30 data_size 0x77(0xbb55 bytes) 2024-09-21T14:22:15.446 INFO:teuthology.orchestra.run.smithi073.stdout:5089 read 0x9d0040b thru 0x9d039c0 (0x35b6 bytes) 2024-09-21T14:22:15.446 INFO:teuthology.orchestra.run.smithi073.stdout:5092 read 0x42ddc2 thru 0x4356ea (0x7929 bytes) 2024-09-21T14:22:15.456 INFO:teuthology.orchestra.run.smithi073.stdout:5094 trunc from 0xcf5f01a to 0x429ac0e 2024-09-21T14:22:15.478 INFO:teuthology.orchestra.run.smithi073.stdout:5096 write 0x7e14d23 thru 0x7e1c006 (0x72e4 bytes) 2024-09-21T14:22:15.480 INFO:teuthology.orchestra.run.smithi073.stdout:5098 read 0x5d8da21 thru 0x5d91a95 (0x4075 bytes) 2024-09-21T14:22:15.491 INFO:teuthology.orchestra.run.smithi073.stdout:5099 write 0xc01ff87 thru 0xc02c557 (0xc5d1 bytes) 2024-09-21T14:22:15.495 INFO:teuthology.orchestra.run.smithi073.stdout:5100 punch from 0xbad1711 to 0xbadfb94, (0xe483 bytes) 2024-09-21T14:22:15.499 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:15.498+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:15.506 INFO:teuthology.orchestra.run.smithi073.stdout:5101 trunc from 0xc02c558 to 0x2186a5e 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:2577 3.b 3:d08ad20f:::rbd_data.11219861e469.0000000000000003:5 [sparse-read 18677760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:15.525 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 51.285900812999998, 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018385483000000001, 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 2577 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238415+0000", 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238417+0000", 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3020000000000002e-06 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238423+0000", 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7950000000000004e-06 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.238427+0000", 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7859999999999996e-06 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256682+0000", 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018254264999999999 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256744+0000", 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1801e-05 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:24.256800+0000", 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6876000000000002e-05 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9533 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927919449000001, 2024-09-21T14:22:15.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018322130999999998, 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9533 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596396+0000", 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596398+0000", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596405+0000", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8990000000000004e-06 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596409+0000", 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.544e-06 2024-09-21T14:22:15.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614559+0000", 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018149477000000001 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614638+0000", 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8975999999999997e-05 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614718+0000", 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0476000000000007e-05 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927902123000003, 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.535 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:15.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.539 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927884609000003, 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:15.543 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927867354999997, 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:15.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:15.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.548 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927839505000001, 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:15.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927817900999997, 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.553 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:15.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.557 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927800902999998, 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:15.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927783883000004, 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:15.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:15.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927766433000002, 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927749294999998, 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927726067999998, 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927709794000002, 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927685422000003, 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927662855000001, 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927645933999997, 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927629435, 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927608438, 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.927586380000001, 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 22.753515508, 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:15.617 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:15.619 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:15.618+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:15.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:15.620 INFO:teuthology.orchestra.run.smithi073.stdout:5103 writesame 0x968a60c thru 0x9693b9b data_size 0x8(0x9590 bytes) 2024-09-21T14:22:15.620 INFO:teuthology.orchestra.run.smithi073.stdout:5104 punch from 0x5487a97 to 0x54891d7, (0x1740 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5105 write 0xcc79b6b thru 0xcc7a290 (0x726 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5106 write 0xbad5356 thru 0xbad6f79 (0x1c24 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5107 trunc from 0xcc7a291 to 0x5eb1ce4 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5109 read 0x1ed59b1 thru 0x1edb739 (0x5d89 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5110 write 0xa260d54 thru 0xa26c919 (0xbbc6 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5111 write 0x7ce8340 thru 0x7cebe6a (0x3b2b bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5112 read 0x86da91d thru 0x86e7483 (0xcb67 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5115 read 0xa4d260 thru 0xa59e46 (0xcbe7 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5116 read 0x8817cdf thru 0x881829a (0x5bc bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5117 trunc from 0xa26c91a to 0xa1bec19 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5118 read 0x3711cda thru 0x3714d81 (0x30a8 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5120 read 0x8c1a76a thru 0x8c2121f (0x6ab6 bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5121 write 0xb67af1b thru 0xb67d2e7 (0x23cd bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5123 writesame 0xcfd8d0b thru 0xcfd9876 data_size 0x22(0xb6c bytes) 2024-09-21T14:22:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:5124 write 0xc38780d thru 0xc391e36 (0xa62a bytes) 2024-09-21T14:22:15.624 INFO:teuthology.orchestra.run.smithi073.stdout:5125 trunc from 0xcfd9877 to 0x27ed52c 2024-09-21T14:22:15.637 INFO:teuthology.orchestra.run.smithi073.stdout:5126 read 0x1306f50 thru 0x130c488 (0x5539 bytes) 2024-09-21T14:22:15.645 INFO:teuthology.orchestra.run.smithi073.stdout:5128 writesame 0x4a7ca60 thru 0x4a843cd data_size 0x42(0x796e bytes) 2024-09-21T14:22:15.655 INFO:teuthology.orchestra.run.smithi073.stdout:5129 write 0x77464ff thru 0x774c230 (0x5d32 bytes) 2024-09-21T14:22:15.658 INFO:teuthology.orchestra.run.smithi073.stdout:5132 write 0x461f0dc thru 0x462ead2 (0xf9f7 bytes) 2024-09-21T14:22:15.662 INFO:teuthology.orchestra.run.smithi073.stdout:5133 read 0x3b4b671 thru 0x3b4c45a (0xdea bytes) 2024-09-21T14:22:15.662 INFO:teuthology.orchestra.run.smithi073.stdout:5136 punch from 0x1fd80e9 to 0x1fe26c1, (0xa5d8 bytes) 2024-09-21T14:22:15.662 INFO:teuthology.orchestra.run.smithi073.stdout:5138 write 0xc8e22 thru 0xd568c (0xc86b bytes) 2024-09-21T14:22:15.664 INFO:teuthology.orchestra.run.smithi073.stdout:5139 punch from 0x5e7f5fa to 0x5e88e71, (0x9877 bytes) 2024-09-21T14:22:15.664 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.677 INFO:teuthology.orchestra.run.smithi073.stdout:5144 writesame 0xc9cde24 thru 0xc9da112 data_size 0x1(0xc2ef bytes) 2024-09-21T14:22:15.677 INFO:teuthology.orchestra.run.smithi073.stdout:5145 writesame 0x155bc1b thru 0x15606b5 data_size 0x1(0x4a9b bytes) 2024-09-21T14:22:15.686 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.686 INFO:teuthology.orchestra.run.smithi073.stdout:5148 punch from 0xb7c956d to 0xb7d23d2, (0x8e65 bytes) 2024-09-21T14:22:15.686 INFO:teuthology.orchestra.run.smithi073.stdout:5149 punch from 0x33c71f8 to 0x33ca569, (0x3371 bytes) 2024-09-21T14:22:15.687 INFO:teuthology.orchestra.run.smithi073.stdout:5150 write 0x5e5f3e2 thru 0x5e64f0e (0x5b2d bytes) 2024-09-21T14:22:15.692 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.692 INFO:teuthology.orchestra.run.smithi073.stdout:5152 punch from 0x7130127 to 0x7139a89, (0x9962 bytes) 2024-09-21T14:22:15.692 INFO:teuthology.orchestra.run.smithi073.stdout:5156 read 0xa100f6e thru 0xa1028bb (0x194e bytes) 2024-09-21T14:22:15.692 INFO:teuthology.orchestra.run.smithi073.stdout:5157 trunc from 0xc9da113 to 0xcbd9bda 2024-09-21T14:22:15.697 INFO:teuthology.orchestra.run.smithi073.stdout:5158 writesame 0xa92597d thru 0xa9284d9 data_size 0x1(0x2b5d bytes) 2024-09-21T14:22:15.712 INFO:teuthology.orchestra.run.smithi073.stdout:5160 writesame 0xe9a3c56 thru 0xe9a9962 data_size 0x7(0x5d0d bytes) 2024-09-21T14:22:15.712 INFO:teuthology.orchestra.run.smithi073.stdout:5161 writesame 0xc13a7b5 thru 0xc149c45 data_size 0x1(0xf491 bytes) 2024-09-21T14:22:15.719 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:15.718+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:15.733 INFO:teuthology.orchestra.run.smithi073.stdout:5162 read 0xa17c59d thru 0xa188a6d (0xc4d1 bytes) 2024-09-21T14:22:15.733 INFO:teuthology.orchestra.run.smithi073.stdout:5163 read 0xcedf23b thru 0xcee2b5a (0x3920 bytes) 2024-09-21T14:22:15.733 INFO:teuthology.orchestra.run.smithi073.stdout:5164 write 0x659b63c thru 0x659f183 (0x3b48 bytes) 2024-09-21T14:22:15.736 INFO:teuthology.orchestra.run.smithi073.stdout:5165 write 0x1263dbf thru 0x126b364 (0x75a6 bytes) 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:75914 3.3 3:c3ccc2b4:::rbd_data.112171459dec.0000000000000009:head [stat out=16b,write 1260991~30118 in=30118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:15.737546+0000", 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.00020964099999999999, 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000240812, 2024-09-21T14:22:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75914 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737546+0000", 2024-09-21T14:22:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737546+0000", 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737545+0000", 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737574+0000", 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8903000000000001e-05 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737585+0000", 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0417999999999999e-05 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737590+0000", 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9169999999999997e-06 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737621+0000", 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1442000000000002e-05 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737666+0000", 2024-09-21T14:22:15.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5300000000000003e-05 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:15.737789+0000", 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012321700000000001 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:22:15.742 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:15.743 INFO:teuthology.orchestra.run.smithi073.stdout:5166 read 0xab89711 thru 0xab8b2b9 (0x1ba9 bytes) 2024-09-21T14:22:15.743 INFO:teuthology.orchestra.run.smithi073.stdout:5168 trunc from 0xe9a9963 to 0xd84260e 2024-09-21T14:22:15.747 INFO:teuthology.orchestra.run.smithi073.stdout:5169 punch from 0xa2fc6bf to 0xa2ffbb1, (0x34f2 bytes) 2024-09-21T14:22:15.747 INFO:teuthology.orchestra.run.smithi073.stdout:5170 read 0xa8e6783 thru 0xa8f63a9 (0xfc27 bytes) 2024-09-21T14:22:15.747 INFO:teuthology.orchestra.run.smithi073.stdout:5172 read 0x916d97 thru 0x91f3d7 (0x8641 bytes) 2024-09-21T14:22:15.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:15.748 INFO:teuthology.orchestra.run.smithi073.stdout:5173 read 0x8496476 thru 0x849c44b (0x5fd6 bytes) 2024-09-21T14:22:15.748 INFO:teuthology.orchestra.run.smithi073.stdout:5174 writesame 0x138d4fa thru 0x1396964 data_size 0x55(0x946b bytes) 2024-09-21T14:22:15.749 INFO:teuthology.orchestra.run.smithi073.stdout:5175 write 0x65a7b7d thru 0x65afad4 (0x7f58 bytes) 2024-09-21T14:22:15.753 INFO:teuthology.orchestra.run.smithi073.stdout:5178 read 0x6b48047 thru 0x6b4b130 (0x30ea bytes) 2024-09-21T14:22:15.754 INFO:teuthology.orchestra.run.smithi073.stdout:5180 read 0x865c743 thru 0x8665fe8 (0x98a6 bytes) 2024-09-21T14:22:15.754 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:15.754 INFO:teuthology.orchestra.run.smithi073.stdout:5182 read 0x296bc65 thru 0x2971eca (0x6266 bytes) 2024-09-21T14:22:15.754 INFO:teuthology.orchestra.run.smithi073.stdout:5183 clone 38 order 20 su 131072 sc 10 2024-09-21T14:22:15.820 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:15.820+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:15.851 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:15.851 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:15.851 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:15.851 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:15.851 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:15.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:15.944 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:15.944+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:16.003 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:16.003 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:16.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43037 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620356230999999, 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044045162999999998, 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43037 2024-09-21T14:22:16.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382704+0000", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1880000000000001e-06 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382710+0000", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.507e-06 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382714+0000", 2024-09-21T14:22:16.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2030000000000002e-06 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426582+0000", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043868468000000001 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426657+0000", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4577000000000001e-05 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426748+0000", 2024-09-21T14:22:16.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.056e-05 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43038 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 319488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620340347999999, 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044219614999999997, 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.008 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43038 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382724+0000", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5659999999999997e-06 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382729+0000", 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-09-21T14:22:16.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382733+0000", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426770+0000", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044036868 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426847+0000", 2024-09-21T14:22:16.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7068000000000001e-05 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426938+0000", 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0708e-05 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43039 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620294233999999, 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044367363999999999, 2024-09-21T14:22:16.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43039 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382767+0000", 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5959999999999998e-06 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382773+0000", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6710000000000002e-06 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382779+0000", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426961+0000", 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044182402000000003 2024-09-21T14:22:16.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427041+0000", 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9810999999999999e-05 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427132+0000", 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1025000000000003e-05 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43040 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620275986999999, 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044540541000000003, 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43040 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382782+0000", 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382784+0000", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0209999999999998e-06 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382790+0000", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5130000000000004e-06 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382794+0000", 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2200000000000003e-06 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427155+0000", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044360978000000002 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427234+0000", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8580999999999999e-05 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427323+0000", 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9584000000000002e-05 2024-09-21T14:22:16.020 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43041 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 354304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620260724000001, 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044711682000000003, 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.021 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43041 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382799+0000", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382805+0000", 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7259999999999999e-06 2024-09-21T14:22:16.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382810+0000", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2910000000000001e-06 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427346+0000", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044536282000000003 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:22:16.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4651000000000001e-05 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427510+0000", 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9147999999999994e-05 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43042 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 364544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620243797000001, 2024-09-21T14:22:16.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044890194000000001, 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43042 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382814+0000", 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382816+0000", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9259999999999999e-06 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382822+0000", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.733e-06 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382826+0000", 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.929e-06 2024-09-21T14:22:16.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427536+0000", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044709567999999998 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427613+0000", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7390000000000003e-05 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427705+0000", 2024-09-21T14:22:16.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1984000000000006e-05 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43043 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620228758, 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045064142000000001, 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.030 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43043 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382831+0000", 2024-09-21T14:22:16.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.874e-06 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382837+0000", 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8320000000000002e-06 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382841+0000", 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1500000000000001e-06 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427728+0000", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044886479 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427805+0000", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.661e-05 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427894+0000", 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9488999999999996e-05 2024-09-21T14:22:16.034 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43044 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620213405000001, 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045458112000000002, 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43044 2024-09-21T14:22:16.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382847+0000", 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9810000000000002e-06 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382853+0000", 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7880000000000003e-06 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382857+0000", 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9790000000000004e-06 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427917+0000", 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045060351999999998 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428174+0000", 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025663200000000002 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428303+0000", 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129701 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43045 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620198054999999, 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045669176999999998, 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43045 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382860+0000", 2024-09-21T14:22:16.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382862+0000", 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382868+0000", 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6500000000000001e-06 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382873+0000", 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5570000000000003e-06 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428330+0000", 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045456929 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428406+0000", 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5184000000000004e-05 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428530+0000", 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124213 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43046 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620181418000001, 2024-09-21T14:22:16.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873328999999997, 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43046 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382879+0000", 2024-09-21T14:22:16.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9800000000000001e-06 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382884+0000", 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5929999999999996e-06 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382888+0000", 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9249999999999997e-06 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428559+0000", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045670193999999997 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428632+0000", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2978999999999994e-05 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428751+0000", 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119015 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.048 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43047 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620166241, 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046080838999999998, 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43047 2024-09-21T14:22:16.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382894+0000", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.956e-06 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382900+0000", 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.541e-06 2024-09-21T14:22:16.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382904+0000", 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0230000000000003e-06 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428777+0000", 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873239000000003 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428850+0000", 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2762000000000006e-05 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428973+0000", 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012365400000000001 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43048 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.053 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620151302, 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046302055000000002, 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43048 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:16.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382909+0000", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.835e-06 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382927+0000", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8580000000000002e-05 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382932+0000", 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5419999999999999e-06 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429004+0000", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046071809999999998 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429077+0000", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3615999999999994e-05 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429209+0000", 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199699999999999 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.057 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43049 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620122931000001, 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492449999999998, 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43049 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382935+0000", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382937+0000", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382943+0000", 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382947+0000", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0670000000000002e-06 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429235+0000", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046287933000000003 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429308+0000", 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2879000000000005e-05 2024-09-21T14:22:16.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429428+0000", 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120269 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43050 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.620107660999999, 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046740351999999999, 2024-09-21T14:22:16.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43050 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382968+0000", 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7289e-05 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383094+0000", 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012638999999999999 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383114+0000", 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0081999999999998e-05 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429454+0000", 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046339578999999999 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429527+0000", 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3450999999999999e-05 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429691+0000", 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016386100000000001 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43051 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619877842000001, 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046543382000000001, 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43051 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383180+0000", 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383184+0000", 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7510000000000002e-06 2024-09-21T14:22:16.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383207+0000", 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2908000000000002e-05 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383231+0000", 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3981999999999998e-05 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429717+0000", 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046485897999999998 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429724+0000", 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5039999999999996e-06 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43052 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:16.071 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619793743999999, 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046500398999999998, 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43052 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383264+0000", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383281+0000", 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6515e-05 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383287+0000", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4439999999999996e-06 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383294+0000", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1680000000000001e-06 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429747+0000", 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046453853000000003 2024-09-21T14:22:16.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429765+0000", 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7918999999999999e-05 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43053 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619759454, 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492406, 2024-09-21T14:22:16.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43053 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383301+0000", 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3580000000000001e-06 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383307+0000", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9619999999999998e-06 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383311+0000", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429785+0000", 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046474161 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429792+0000", 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.139e-06 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43054 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619743676999999, 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046502169000000003, 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43054 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383317+0000", 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9709999999999998e-06 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383322+0000", 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5589999999999996e-06 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383328+0000", 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046481947000000003 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429817+0000", 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43055 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619726580999998, 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046513090999999999, 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43055 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:16.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383337+0000", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383342+0000", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4559999999999999e-06 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383347+0000", 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3830000000000002e-06 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429839+0000", 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492377000000001 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429845+0000", 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1689999999999997e-06 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.087 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43056 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.619708229, 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046522758999999997, 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43056 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383352+0000", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383357+0000", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5380000000000002e-06 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383362+0000", 2024-09-21T14:22:16.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2629999999999997e-06 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429865+0000", 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046503567000000003 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429873+0000", 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.858e-06 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.092 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:16.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:16.094 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:16.093+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:16.194 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:16.194+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:16.235 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:16.235 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:16.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:16.295 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:16.295+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:16.346 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:16.346 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:16.346 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:16.346 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:16.346 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:16.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:16.399 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:16.399+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.238500430000002, 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:16.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:16.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:16.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42293 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3274752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.396085430999999, 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032075181000000001, 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42293 2024-09-21T14:22:16.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063351+0000", 2024-09-21T14:22:16.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063354+0000", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0880000000000002e-06 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063359+0000", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7620000000000001e-06 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063364+0000", 2024-09-21T14:22:16.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3560000000000003e-06 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095339+0000", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031974925000000001 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095389+0000", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0095000000000001e-05 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095427+0000", 2024-09-21T14:22:16.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8389000000000001e-05 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42294 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3500032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.396069826000002, 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032160279, 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.469 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42294 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063369+0000", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0320000000000002e-06 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063375+0000", 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4729999999999999e-06 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063379+0000", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5329999999999996e-06 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095440+0000", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032060937999999997 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095489+0000", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8711000000000002e-05 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095528+0000", 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8926e-05 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53665 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746709373000002, 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032083390000000003, 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53665 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712733+0000", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712740+0000", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9290000000000001e-06 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712746+0000", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744674+0000", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031927981000000001 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744749+0000", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744811+0000", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2681000000000005e-05 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746675968000002, 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.477 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:16.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:16.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:16.482 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746658264000001, 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:16.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:16.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746642593000001, 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.490 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746627079, 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.494 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746612012, 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746596817, 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.499 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:16.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:16.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746581711000001, 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:16.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746566601000001, 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:16.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:16.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746551235999998, 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:16.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:16.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.515 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746535776999998, 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746520744000001, 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.520 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:16.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746504855000001, 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.524 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:16.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:16.528 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746479179000001, 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:16.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:16.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746463894000001, 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.533 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:16.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:16.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:16.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.74644193, 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:16.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:16.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.746425946999999, 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:16.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:16.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:16.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:16.545 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:16.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:22:16.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:16.547 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:16.546+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:16.553 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone37 from 0xd84260e (overlap 0x1502e4) to 0x8678e 2024-09-21T14:22:16.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:16.643 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T14:22:16.647 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:16.646+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:16.667 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:16.668 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:16.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:16.747 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:16.747+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:16.753 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:16.771 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:16.772 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:16.805 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:16.805 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:16.822 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:16.822 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:16.831 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:16.832 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:16.839 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:16.839 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:16.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:16.848 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:16.847+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:16.856 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:16.856 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:16.873 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:16.873 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:16.874 INFO:teuthology.orchestra.run.smithi073.stdout:5185 punch from 0x1e470e7 to 0x1e47880, (0x799 bytes) 2024-09-21T14:22:16.878 INFO:teuthology.orchestra.run.smithi073.stdout:5186 punch from 0x5b797fa to 0x5b884e2, (0xece8 bytes) 2024-09-21T14:22:16.879 INFO:teuthology.orchestra.run.smithi073.stdout:5188 punch from 0xd49cc0e to 0xd49e0f3, (0x14e5 bytes) 2024-09-21T14:22:16.879 INFO:teuthology.orchestra.run.smithi073.stdout:5190 read 0xd7f25cd thru 0xd7fa8d1 (0x8305 bytes) 2024-09-21T14:22:16.879 INFO:teuthology.orchestra.run.smithi073.stdout:5191 read 0x7cec568 thru 0x7cfacc9 (0xe762 bytes) 2024-09-21T14:22:16.879 INFO:teuthology.orchestra.run.smithi073.stdout:5192 write 0x47b0667 thru 0x47b4886 (0x4220 bytes) 2024-09-21T14:22:16.883 INFO:teuthology.orchestra.run.smithi073.stdout:5193 write 0x1175fa9 thru 0x1181c06 (0xbc5e bytes) 2024-09-21T14:22:16.894 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:16.894 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:16.896 INFO:teuthology.orchestra.run.smithi073.stdout:5194 write 0xeca7668 thru 0xecaf351 (0x7cea bytes) 2024-09-21T14:22:16.899 INFO:teuthology.orchestra.run.smithi073.stdout:5195 read 0x4b39188 thru 0x4b470ae (0xdf27 bytes) 2024-09-21T14:22:16.900 INFO:teuthology.orchestra.run.smithi073.stdout:5196 punch from 0x77148e3 to 0x771a002, (0x571f bytes) 2024-09-21T14:22:16.900 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:16.903 INFO:teuthology.orchestra.run.smithi073.stdout:5198 trunc from 0xecaf352 to 0xc462b14 2024-09-21T14:22:16.923 INFO:teuthology.orchestra.run.smithi073.stdout:5199 write 0x8cba34d thru 0x8cbcde7 (0x2a9b bytes) 2024-09-21T14:22:16.934 INFO:teuthology.orchestra.run.smithi073.stdout:5200 write 0xe3473d6 thru 0xe349b85 (0x27b0 bytes) 2024-09-21T14:22:16.937 INFO:teuthology.orchestra.run.smithi073.stdout:5201 writesame 0xa918bae thru 0xa9236a1 data_size 0x1c(0xaaf4 bytes) 2024-09-21T14:22:16.941 INFO:teuthology.orchestra.run.smithi073.stdout:5202 punch from 0x76f9a95 to 0x76fbe86, (0x23f1 bytes) 2024-09-21T14:22:16.941 INFO:teuthology.orchestra.run.smithi073.stdout:5203 write 0xc0007da thru 0xc00f912 (0xf139 bytes) 2024-09-21T14:22:16.948 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:16.948+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:16.949 INFO:teuthology.orchestra.run.smithi073.stdout:5205 write 0xeb9189b thru 0xeb95178 (0x38de bytes) 2024-09-21T14:22:16.952 INFO:teuthology.orchestra.run.smithi073.stdout:5206 punch from 0x9a59f18 to 0x9a64831, (0xa919 bytes) 2024-09-21T14:22:16.952 INFO:teuthology.orchestra.run.smithi073.stdout:5207 read 0x8dd05ce thru 0x8dd9782 (0x91b5 bytes) 2024-09-21T14:22:16.953 INFO:teuthology.orchestra.run.smithi073.stdout:5208 read 0xdb798cb thru 0xdb7b23d (0x1973 bytes) 2024-09-21T14:22:16.953 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:16.953 INFO:teuthology.orchestra.run.smithi073.stdout:5210 read 0xa3a7a0b thru 0xa3aa8c3 (0x2eb9 bytes) 2024-09-21T14:22:16.955 INFO:teuthology.orchestra.run.smithi073.stdout:5211 trunc from 0xeb95179 to 0xc20e75e 2024-09-21T14:22:16.966 INFO:teuthology.orchestra.run.smithi073.stdout:5212 read 0x2360e97 thru 0x236ff8b (0xf0f5 bytes) 2024-09-21T14:22:16.966 INFO:teuthology.orchestra.run.smithi073.stdout:5213 read 0x54fa8f6 thru 0x54ff10c (0x4817 bytes) 2024-09-21T14:22:16.971 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:16.972 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:16.976 INFO:teuthology.orchestra.run.smithi073.stdout:5214 trunc from 0xc20e75e to 0x4b52475 2024-09-21T14:22:16.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:17.012 INFO:teuthology.orchestra.run.smithi073.stdout:5215 trunc from 0x4b52475 to 0xd738714 2024-09-21T14:22:17.018 INFO:teuthology.orchestra.run.smithi073.stdout:5216 read 0x536c9b7 thru 0x5379f1f (0xd569 bytes) 2024-09-21T14:22:17.018 INFO:teuthology.orchestra.run.smithi073.stdout:5217 punch from 0x6a5c620 to 0x6a62da9, (0x6789 bytes) 2024-09-21T14:22:17.018 INFO:teuthology.orchestra.run.smithi073.stdout:5219 read 0x91e84c9 thru 0x91eb73b (0x3273 bytes) 2024-09-21T14:22:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:5220 writesame 0xe854bf5 thru 0xe8602a6 data_size 0x2(0xb6b2 bytes) 2024-09-21T14:22:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:5221 read 0x632a64a thru 0x632a909 (0x2c0 bytes) 2024-09-21T14:22:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:5225 read 0xb05ab1b thru 0xb068587 (0xda6d bytes) 2024-09-21T14:22:17.034 INFO:teuthology.orchestra.run.smithi073.stdout:5226 trunc from 0xe8602a7 to 0x6aa3580 2024-09-21T14:22:17.039 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:5228 write 0x8aa18d1 thru 0x8aafd9c (0xe4cc bytes) 2024-09-21T14:22:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:5229 read 0x5c1f92d thru 0x5c2c11c (0xc7f0 bytes) 2024-09-21T14:22:17.050 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:17.049+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:17.054 INFO:teuthology.orchestra.run.smithi073.stdout:5230 trunc from 0x8aafd9d to 0x53fc005 2024-09-21T14:22:17.063 INFO:teuthology.orchestra.run.smithi073.stdout:5233 punch from 0x16cba3d to 0x16d1975, (0x5f38 bytes) 2024-09-21T14:22:17.078 INFO:teuthology.orchestra.run.smithi073.stdout:5234 writesame 0xcd51b95 thru 0xcd54866 data_size 0x1(0x2cd2 bytes) 2024-09-21T14:22:17.080 INFO:teuthology.orchestra.run.smithi073.stdout:5235 trunc from 0xcd54867 to 0xac62c59 2024-09-21T14:22:17.086 INFO:teuthology.orchestra.run.smithi073.stdout:5236 writesame 0x1160f2a thru 0x116d208 data_size 0x45(0xc2df bytes) 2024-09-21T14:22:17.088 INFO:teuthology.orchestra.run.smithi073.stdout:5237 punch from 0xa2073ca to 0xa2165d5, (0xf20b bytes) 2024-09-21T14:22:17.088 INFO:teuthology.orchestra.run.smithi073.stdout:5239 read 0xcaa2f5 thru 0xcb3baa (0x98b6 bytes) 2024-09-21T14:22:17.089 INFO:teuthology.orchestra.run.smithi073.stdout:5240 read 0x75571c3 thru 0x755b2d6 (0x4114 bytes) 2024-09-21T14:22:17.089 INFO:teuthology.orchestra.run.smithi073.stdout:5241 read 0x9beebf2 thru 0x9bfceed (0xe2fc bytes) 2024-09-21T14:22:17.089 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.089 INFO:teuthology.orchestra.run.smithi073.stdout:5243 write 0x6faae49 thru 0x6fb4d36 (0x9eee bytes) 2024-09-21T14:22:17.092 INFO:teuthology.orchestra.run.smithi073.stdout:5245 write 0x3cf7f35 thru 0x3cfc71b (0x47e7 bytes) 2024-09-21T14:22:17.097 INFO:teuthology.orchestra.run.smithi073.stdout:5246 writesame 0xa7aca42 thru 0xa7ad087 data_size 0xb(0x646 bytes) 2024-09-21T14:22:17.100 INFO:teuthology.orchestra.run.smithi073.stdout:5247 writesame 0x1403eba thru 0x140cc85 data_size 0x96(0x8dcc bytes) 2024-09-21T14:22:17.106 INFO:teuthology.orchestra.run.smithi073.stdout:5249 trunc from 0xac62c59 to 0x9998f69 2024-09-21T14:22:17.117 INFO:teuthology.orchestra.run.smithi073.stdout:5250 punch from 0x15507f3 to 0x155b259, (0xaa66 bytes) 2024-09-21T14:22:17.117 INFO:teuthology.orchestra.run.smithi073.stdout:5251 write 0x1fe7597 thru 0x1ff2f12 (0xb97c bytes) 2024-09-21T14:22:17.128 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.128 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.129 INFO:teuthology.orchestra.run.smithi073.stdout:5252 writesame 0xb6a6b22 thru 0xb6a9601 data_size 0x38(0x2ae0 bytes) 2024-09-21T14:22:17.129 INFO:teuthology.orchestra.run.smithi073.stdout:5253 write 0x351e30d thru 0x352bf76 (0xdc6a bytes) 2024-09-21T14:22:17.136 INFO:teuthology.orchestra.run.smithi073.stdout:5254 read 0x5dfff41 thru 0x5e04155 (0x4215 bytes) 2024-09-21T14:22:17.136 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.136 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.137 INFO:teuthology.orchestra.run.smithi073.stdout:5258 clone 39 order 19 su 16384 sc 3 2024-09-21T14:22:17.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:17.179 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:17.179+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:17.252 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:17.252+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:17.279 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.280 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:17.353 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:17.353+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:17.391 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.392 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:17.454 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:17.454+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:17.491 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:22:17.506 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.506 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:17.559 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:17.559+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:17.561 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone38 from 0xb6a9602 (overlap 0x4b52475) to 0x25a923a 2024-09-21T14:22:17.620 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:17.620 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:17.620 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:22:17.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:22:17.635 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.635 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.644 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:17.664 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:17.664+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:17.691 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-21T14:22:17.695 INFO:teuthology.orchestra.run.smithi073.stdout:5259 writesame 0x87f191b thru 0x880101f data_size 0x6b(0xf705 bytes) 2024-09-21T14:22:17.702 INFO:teuthology.orchestra.run.smithi073.stdout:5260 read 0x2ecf762 thru 0x2ed0c97 (0x1536 bytes) 2024-09-21T14:22:17.703 INFO:teuthology.orchestra.run.smithi073.stdout:5261 trunc from 0xb6a9602 to 0xb6ffedb 2024-09-21T14:22:17.716 INFO:teuthology.orchestra.run.smithi073.stdout:5265 write 0xcae7be8 thru 0xcaec65a (0x4a73 bytes) 2024-09-21T14:22:17.737 INFO:teuthology.orchestra.run.smithi073.stdout:5266 trunc from 0xcaec65b to 0x197d1fc 2024-09-21T14:22:17.753 INFO:teuthology.orchestra.run.smithi073.stdout:5268 trunc from 0x197d1fc to 0x6e9ed8 2024-09-21T14:22:17.756 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:17.756+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:17.790 INFO:teuthology.orchestra.run.smithi073.stdout:5269 write 0xc8ec27b thru 0xc8f262d (0x63b3 bytes) 2024-09-21T14:22:17.795 INFO:teuthology.orchestra.run.smithi073.stdout:5270 read 0xaf8aa70 thru 0xaf91092 (0x6623 bytes) 2024-09-21T14:22:17.795 INFO:teuthology.orchestra.run.smithi073.stdout:5271 read 0x1d1a005 thru 0x1d29d76 (0xfd72 bytes) 2024-09-21T14:22:17.795 INFO:teuthology.orchestra.run.smithi073.stdout:5272 read 0xc1b1a40 thru 0xc1bce5c (0xb41d bytes) 2024-09-21T14:22:17.796 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:17.796 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:17.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:17.805 INFO:teuthology.orchestra.run.smithi073.stdout:5274 trunc from 0xc8f262e to 0x4945cf8 2024-09-21T14:22:17.823 INFO:teuthology.orchestra.run.smithi073.stdout:5276 trunc from 0x4945cf8 to 0x38cc7e 2024-09-21T14:22:17.872 INFO:teuthology.orchestra.run.smithi073.stdout:5277 writesame 0xe0929f3 thru 0xe0990cf data_size 0x1(0x66dd bytes) 2024-09-21T14:22:17.872 INFO:teuthology.orchestra.run.smithi073.stdout:5278 writesame 0x498f0f0 thru 0x499b3d3 data_size 0x2(0xc2e4 bytes) 2024-09-21T14:22:17.883 INFO:teuthology.orchestra.run.smithi073.stdout:5281 read 0x448658d thru 0x4486ce3 (0x757 bytes) 2024-09-21T14:22:17.883 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.883 INFO:teuthology.orchestra.run.smithi073.stdout:5284 read 0x54cb1e7 thru 0x54d2c68 (0x7a82 bytes) 2024-09-21T14:22:17.883 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:17.883 INFO:teuthology.orchestra.run.smithi073.stdout:5286 writesame 0x60de0c4 thru 0x60e129c data_size 0x1(0x31d9 bytes) 2024-09-21T14:22:17.888 INFO:teuthology.orchestra.run.smithi073.stdout:5288 writesame 0x12e56b7 thru 0x12f4fc2 data_size 0xa1(0xf90c bytes) 2024-09-21T14:22:17.889 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:17.889+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:17.898 INFO:teuthology.orchestra.run.smithi073.stdout:5289 read 0xc21c5b8 thru 0xc228f9e (0xc9e7 bytes) 2024-09-21T14:22:17.905 INFO:teuthology.orchestra.run.smithi073.stdout:5292 write 0xedc125a thru 0xedc3465 (0x220c bytes) 2024-09-21T14:22:17.910 INFO:teuthology.orchestra.run.smithi073.stdout:5293 trunc from 0xedc3466 to 0xbfff664 2024-09-21T14:22:17.920 INFO:teuthology.orchestra.run.smithi073.stdout:5294 read 0x1192752 thru 0x119c2ed (0x9b9c bytes) 2024-09-21T14:22:17.920 INFO:teuthology.orchestra.run.smithi073.stdout:5295 write 0x37126b8 thru 0x371a291 (0x7bda bytes) 2024-09-21T14:22:17.927 INFO:teuthology.orchestra.run.smithi073.stdout:5296 writesame 0x3e44a38 thru 0x3e4b244 data_size 0xd(0x680d bytes) 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 72, 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:22:17.485573+0000", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:21:48.316234+0000", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5801984, 2024-09-21T14:22:17.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:22:17.934 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:17.935 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:22:17.936 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:17.937 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:17.938 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:17.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:17.940 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.9900000095367432, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.9900000095367432, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.75, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.75, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:22:17.941 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:17.942 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:17.943 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:17.944 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:17.946 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:17.947 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:22:17.948 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "72", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:17.949 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 42, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 72, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:17.950 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:17.951 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 1.7400000095367432, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 1.7400000095367432, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T14:22:17.952 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.953 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.954 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.958 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 33, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.959 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.961 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.962 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:22:17.963 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.964 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.965 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:22:17.966 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.969 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.973 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.975 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.976 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.978 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:22:17.979 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.980 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 65, 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.981 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.984 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.985 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.986 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:22:17.987 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.988 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.989 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:17.990 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:17.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.996 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.998 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:17.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "1.1", 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:18.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.d", 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:22:18.002 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:22:18.003 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:22:18.004 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:22:18.004 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:22:18.004 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:22:18.004 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:18.004 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:18.005 INFO:teuthology.orchestra.run.smithi073.stdout:5297 trunc from 0xbfff664 to 0x1c13c 2024-09-21T14:22:18.006 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.006 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.006 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:18.007 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:18.004+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:18.007 INFO:tasks.thrashosds.thrasher:j is {'epoch': 72, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:22:17.485573+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:21:48.316234+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 72, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.1', 'osds': [5, 4]}, {'pgid': '3.d', 'osds': [6, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:22:18.007 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:5298 trunc from 0x1c13c to 0x85377dd 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:5302 read 0x311ac3 thru 0x31c8d7 (0xae15 bytes) 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:5303 read 0x3e9fcc thru 0x3f480e (0xa843 bytes) 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:5305 read 0x595d206 thru 0x59695e4 (0xc3df bytes) 2024-09-21T14:22:18.008 INFO:teuthology.orchestra.run.smithi073.stdout:5306 read 0x4aadd75 thru 0x4ab84b0 (0xa73c bytes) 2024-09-21T14:22:18.009 INFO:teuthology.orchestra.run.smithi073.stdout:5307 writesame 0x13603e0 thru 0x136d7c8 data_size 0x6b(0xd3e9 bytes) 2024-09-21T14:22:18.009 INFO:teuthology.orchestra.run.smithi073.stdout:5309 trunc from 0x85377dd to 0x311a932 2024-09-21T14:22:18.010 INFO:teuthology.orchestra.run.smithi073.stdout:5311 trunc from 0x311a932 to 0x350be59 2024-09-21T14:22:18.027 INFO:teuthology.orchestra.run.smithi073.stdout:5313 write 0x8f1ae45 thru 0x8f1dec2 (0x307e bytes) 2024-09-21T14:22:18.031 INFO:teuthology.orchestra.run.smithi073.stdout:5314 read 0x779fd1e thru 0x77a3ad5 (0x3db8 bytes) 2024-09-21T14:22:18.031 INFO:teuthology.orchestra.run.smithi073.stdout:5315 write 0x64f7588 thru 0x64f9c79 (0x26f2 bytes) 2024-09-21T14:22:18.034 INFO:teuthology.orchestra.run.smithi073.stdout:5316 punch from 0x147f98f to 0x1480f69, (0x15da bytes) 2024-09-21T14:22:18.034 INFO:teuthology.orchestra.run.smithi073.stdout:5317 write 0x61916d3 thru 0x61921eb (0xb19 bytes) 2024-09-21T14:22:18.042 INFO:teuthology.orchestra.run.smithi073.stdout:5318 trunc from 0x8f1dec3 to 0xd41e129 2024-09-21T14:22:18.056 INFO:teuthology.orchestra.run.smithi073.stdout:5319 trunc from 0xd41e129 to 0x58234b9 2024-09-21T14:22:18.066 INFO:teuthology.orchestra.run.smithi073.stdout:5320 write 0xd93ff2 thru 0xd99b38 (0x5b47 bytes) 2024-09-21T14:22:18.071 INFO:teuthology.orchestra.run.smithi073.stdout:5323 read 0x488f148 thru 0x48901eb (0x10a4 bytes) 2024-09-21T14:22:18.071 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.071 INFO:teuthology.orchestra.run.smithi073.stdout:5328 read 0x4b0f12c thru 0x4b0f2aa (0x17f bytes) 2024-09-21T14:22:18.080 INFO:teuthology.orchestra.run.smithi073.stdout:5329 write 0x8e4647d thru 0x8e476f7 (0x127b bytes) 2024-09-21T14:22:18.084 INFO:teuthology.orchestra.run.smithi073.stdout:5330 read 0x6e1052e thru 0x6e1a1e3 (0x9cb6 bytes) 2024-09-21T14:22:18.084 INFO:teuthology.orchestra.run.smithi073.stdout:5331 read 0x2c7fe82 thru 0x2c7ffb0 (0x12f bytes) 2024-09-21T14:22:18.084 INFO:teuthology.orchestra.run.smithi073.stdout:5333 read 0x2ac8c9f thru 0x2ad0c5d (0x7fbf bytes) 2024-09-21T14:22:18.089 INFO:teuthology.orchestra.run.smithi073.stdout:5334 write 0x9664fca thru 0x9673c20 (0xec57 bytes) 2024-09-21T14:22:18.094 INFO:teuthology.orchestra.run.smithi073.stdout:5335 write 0x784d1d3 thru 0x784f396 (0x21c4 bytes) 2024-09-21T14:22:18.097 INFO:teuthology.orchestra.run.smithi073.stdout:5337 read 0x888d9e1 thru 0x8899b54 (0xc174 bytes) 2024-09-21T14:22:18.097 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.105 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:18.105+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:18.108 INFO:teuthology.orchestra.run.smithi073.stdout:5341 write 0xe4ce439 thru 0xe4d9ef0 (0xbab8 bytes) 2024-09-21T14:22:18.130 INFO:teuthology.orchestra.run.smithi073.stdout:5343 trunc from 0xe4d9ef1 to 0x50c8a4 2024-09-21T14:22:18.148 INFO:teuthology.orchestra.run.smithi073.stdout:5344 write 0x18962a8 thru 0x189bb45 (0x589e bytes) 2024-09-21T14:22:18.156 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.156 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.164 INFO:teuthology.orchestra.run.smithi073.stdout:5346 writesame 0x6d98e59 thru 0x6da025a data_size 0x1f(0x7402 bytes) 2024-09-21T14:22:18.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:18.179 INFO:teuthology.orchestra.run.smithi073.stdout:5347 write 0xe552bf2 thru 0xe55f48e (0xc89d bytes) 2024-09-21T14:22:18.184 INFO:teuthology.orchestra.run.smithi073.stdout:5348 trunc from 0xe55f48f to 0xdf18886 2024-09-21T14:22:18.194 INFO:teuthology.orchestra.run.smithi073.stdout:5350 read 0x9d34fa9 thru 0x9d4231b (0xd373 bytes) 2024-09-21T14:22:18.194 INFO:teuthology.orchestra.run.smithi073.stdout:5352 read 0x687f4e8 thru 0x688e324 (0xee3d bytes) 2024-09-21T14:22:18.194 INFO:teuthology.orchestra.run.smithi073.stdout:5353 punch from 0x26ee7ed to 0x26f1aba, (0x32cd bytes) 2024-09-21T14:22:18.194 INFO:teuthology.orchestra.run.smithi073.stdout:5354 read 0xa939979 thru 0xa93e2d6 (0x495e bytes) 2024-09-21T14:22:18.194 INFO:teuthology.orchestra.run.smithi073.stdout:5355 writesame 0x856679c thru 0x8571269 data_size 0x1(0xaace bytes) 2024-09-21T14:22:18.209 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:18.208+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:18.209 INFO:teuthology.orchestra.run.smithi073.stdout:5356 read 0xced42d6 thru 0xced8c8d (0x49b8 bytes) 2024-09-21T14:22:18.210 INFO:teuthology.orchestra.run.smithi073.stdout:5357 read 0xa50d59f thru 0xa51553d (0x7f9f bytes) 2024-09-21T14:22:18.210 INFO:teuthology.orchestra.run.smithi073.stdout:5358 trunc from 0xdf18886 to 0xdcbd650 2024-09-21T14:22:18.215 INFO:teuthology.orchestra.run.smithi073.stdout:5359 punch from 0x41648d to 0x417260, (0xdd3 bytes) 2024-09-21T14:22:18.215 INFO:teuthology.orchestra.run.smithi073.stdout:5360 write 0x1fb5297 thru 0x1fbc97d (0x76e7 bytes) 2024-09-21T14:22:18.223 INFO:teuthology.orchestra.run.smithi073.stdout:5361 write 0x5a47989 thru 0x5a4f74e (0x7dc6 bytes) 2024-09-21T14:22:18.227 INFO:teuthology.orchestra.run.smithi073.stdout:5362 read 0x21f1b97 thru 0x21ff5bc (0xda26 bytes) 2024-09-21T14:22:18.228 INFO:teuthology.orchestra.run.smithi073.stdout:5363 punch from 0x5e25be6 to 0x5e2ddcf, (0x81e9 bytes) 2024-09-21T14:22:18.228 INFO:teuthology.orchestra.run.smithi073.stdout:5364 write 0x1aed920 thru 0x1af0d95 (0x3476 bytes) 2024-09-21T14:22:18.232 INFO:teuthology.orchestra.run.smithi073.stdout:5365 punch from 0x37298af to 0x3736402, (0xcb53 bytes) 2024-09-21T14:22:18.237 INFO:teuthology.orchestra.run.smithi073.stdout:5366 write 0xe37d481 thru 0xe385c74 (0x87f4 bytes) 2024-09-21T14:22:18.241 INFO:teuthology.orchestra.run.smithi073.stdout:5368 writesame 0xdc874a8 thru 0xdc8f189 data_size 0x73(0x7ce2 bytes) 2024-09-21T14:22:18.250 INFO:teuthology.orchestra.run.smithi073.stdout:5369 read 0x804aea8 thru 0x804d3e4 (0x253d bytes) 2024-09-21T14:22:18.250 INFO:teuthology.orchestra.run.smithi073.stdout:5370 writesame 0x37bc143 thru 0x37c5689 data_size 0x1(0x9547 bytes) 2024-09-21T14:22:18.262 INFO:teuthology.orchestra.run.smithi073.stdout:5371 write 0xc2e5543 thru 0xc2e68c8 (0x1386 bytes) 2024-09-21T14:22:18.265 INFO:teuthology.orchestra.run.smithi073.stdout:5372 writesame 0x206f48a thru 0x207ec04 data_size 0x1(0xf77b bytes) 2024-09-21T14:22:18.282 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.283 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.286 INFO:teuthology.orchestra.run.smithi073.stdout:5373 trunc from 0xe385c75 to 0x7206bb1 2024-09-21T14:22:18.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:18.299 INFO:teuthology.orchestra.run.smithi073.stdout:5374 read 0x17313bd thru 0x1736f04 (0x5b48 bytes) 2024-09-21T14:22:18.299 INFO:teuthology.orchestra.run.smithi073.stdout:5375 read 0x6c47651 thru 0x6c54074 (0xca24 bytes) 2024-09-21T14:22:18.299 INFO:teuthology.orchestra.run.smithi073.stdout:5376 write 0xed6730 thru 0xedc4c2 (0x5d93 bytes) 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:5378 punch from 0x504ca34 to 0x505292d, (0x5ef9 bytes) 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:5379 punch from 0x6170ea2 to 0x617abe7, (0x9d45 bytes) 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:5381 read 0x4ff26eb thru 0x4ff5e7b (0x3791 bytes) 2024-09-21T14:22:18.307 INFO:teuthology.orchestra.run.smithi073.stdout:5383 punch from 0x190009e to 0x190fb75, (0xfad7 bytes) 2024-09-21T14:22:18.308 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:18.307+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:18.308 INFO:teuthology.orchestra.run.smithi073.stdout:5387 read 0x6eda85e thru 0x6ee0640 (0x5de3 bytes) 2024-09-21T14:22:18.308 INFO:teuthology.orchestra.run.smithi073.stdout:5388 writesame 0x55dad52 thru 0x55e5b71 data_size 0x38(0xae20 bytes) 2024-09-21T14:22:18.311 INFO:teuthology.orchestra.run.smithi073.stdout:5391 read 0x2dd39f6 thru 0x2ddecae (0xb2b9 bytes) 2024-09-21T14:22:18.311 INFO:teuthology.orchestra.run.smithi073.stdout:5393 write 0x10bd0ab thru 0x10c15d5 (0x452b bytes) 2024-09-21T14:22:18.314 INFO:teuthology.orchestra.run.smithi073.stdout:5394 punch from 0x53ab05d to 0x53b88c8, (0xd86b bytes) 2024-09-21T14:22:18.314 INFO:teuthology.orchestra.run.smithi073.stdout:5395 write 0x55f34d3 thru 0x55fcf15 (0x9a43 bytes) 2024-09-21T14:22:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:5397 punch from 0xbc0dd2 to 0xbc5f89, (0x51b7 bytes) 2024-09-21T14:22:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:5398 read 0x174d93c thru 0x1754f83 (0x7648 bytes) 2024-09-21T14:22:18.316 INFO:teuthology.orchestra.run.smithi073.stdout:5399 read 0x58d97fb thru 0x58e2a6f (0x9275 bytes) 2024-09-21T14:22:18.321 INFO:teuthology.orchestra.run.smithi073.stdout:5400 trunc from 0x7206bb1 to 0x4c70db4 2024-09-21T14:22:18.331 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:18.331 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:22:18.332 INFO:teuthology.orchestra.run.smithi073.stdout:5401 trunc from 0x4c70db4 to 0x65fab65 2024-09-21T14:22:18.337 INFO:teuthology.orchestra.run.smithi073.stdout:5402 read 0x19f3548 thru 0x19fd846 (0xa2ff bytes) 2024-09-21T14:22:18.345 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-21T14:22:17.489872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1359348207,"num_objects":1707,"num_object_clones":461,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":682,"num_read":112436,"num_read_kb":535590,"num_write":16356,"num_write_kb":265357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13311,"ondisk_log_size":13311,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":318952,"kb_used_data":48676,"kb_used_omap":10,"kb_used_meta":270197,"kb_avail":660283928,"statfs":{"total":676457349120,"available":676130742272,"internally_reserved":0,"allocated":49844224,"data_stored":382568196,"data_compressed":24037731,"data_compressed_allocated":25985024,"data_compressed_original":364960875,"omap_allocated":11150,"internal_metadata":276681842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":142121775,"num_objects":219,"num_object_clones":86,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":49,"num_read":16547,"num_read_kb":59874,"num_write":3667,"num_write_kb":27033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.061783"},"pg_stats":[{"pgid":"3.1f","version":"71'275","reported_seq":2365,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.131059+0000","last_change":"2024-09-21T14:21:20.149669+0000","last_active":"2024-09-21T14:22:17.131059+0000","last_peered":"2024-09-21T14:22:17.131059+0000","last_clean":"2024-09-21T14:22:17.131059+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:22:17.131059+0000","last_undegraded":"2024-09-21T14:22:17.131059+0000","last_fullsized":"2024-09-21T14:22:17.131059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:23.084910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":21747151,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":3066,"num_read_kb":14945,"num_write":280,"num_write_kb":4645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"71'216","reported_seq":1086,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.987231+0000","last_change":"2024-09-21T14:21:49.388656+0000","last_active":"2024-09-21T14:22:16.987231+0000","last_peered":"2024-09-21T14:22:16.987231+0000","last_clean":"2024-09-21T14:22:16.987231+0000","last_became_active":"2024-09-21T14:21:49.343683+0000","last_became_peered":"2024-09-21T14:21:49.343683+0000","last_unstale":"2024-09-21T14:22:16.987231+0000","last_undegraded":"2024-09-21T14:22:16.987231+0000","last_fullsized":"2024-09-21T14:22:16.987231+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:13.419608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":32173359,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":815,"num_read_kb":9718,"num_write":196,"num_write_kb":6259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2693260,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"71'342","reported_seq":2347,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.982605+0000","last_change":"2024-09-21T14:21:20.161845+0000","last_active":"2024-09-21T14:22:16.982605+0000","last_peered":"2024-09-21T14:22:16.982605+0000","last_clean":"2024-09-21T14:22:16.982605+0000","last_became_active":"2024-09-21T14:21:18.145217+0000","last_became_peered":"2024-09-21T14:21:18.145217+0000","last_unstale":"2024-09-21T14:22:16.982605+0000","last_undegraded":"2024-09-21T14:22:16.982605+0000","last_fullsized":"2024-09-21T14:22:16.982605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:33.663662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":21661609,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":3343,"num_read_kb":15302,"num_write":373,"num_write_kb":8175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"71'269","reported_seq":4149,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.130930+0000","last_change":"2024-09-21T14:21:20.143875+0000","last_active":"2024-09-21T14:22:17.130930+0000","last_peered":"2024-09-21T14:22:17.130930+0000","last_clean":"2024-09-21T14:22:17.130930+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:22:17.130930+0000","last_undegraded":"2024-09-21T14:22:17.130930+0000","last_fullsized":"2024-09-21T14:22:17.130930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:02.708302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":48341846,"num_objects":68,"num_object_clones":21,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":21,"num_read":4025,"num_read_kb":32409,"num_write":263,"num_write_kb":7456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"71'361","reported_seq":3490,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.131438+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:22:17.131438+0000","last_peered":"2024-09-21T14:22:17.131438+0000","last_clean":"2024-09-21T14:22:17.131438+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:22:17.131438+0000","last_undegraded":"2024-09-21T14:22:17.131438+0000","last_fullsized":"2024-09-21T14:22:17.131438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":88864407,"num_objects":61,"num_object_clones":20,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":4300,"num_read_kb":38127,"num_write":328,"num_write_kb":18894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"71'373","reported_seq":3827,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.129582+0000","last_change":"2024-09-21T14:21:20.143780+0000","last_active":"2024-09-21T14:22:17.129582+0000","last_peered":"2024-09-21T14:22:17.129582+0000","last_clean":"2024-09-21T14:22:17.129582+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:22:17.129582+0000","last_undegraded":"2024-09-21T14:22:17.129582+0000","last_fullsized":"2024-09-21T14:22:17.129582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:03.668551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":37845816,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":5383,"num_read_kb":9080,"num_write":416,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"71'446","reported_seq":4742,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.938521+0000","last_change":"2024-09-21T14:21:20.146200+0000","last_active":"2024-09-21T14:22:16.938521+0000","last_peered":"2024-09-21T14:22:16.938521+0000","last_clean":"2024-09-21T14:22:16.938521+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:22:16.938521+0000","last_undegraded":"2024-09-21T14:22:16.938521+0000","last_fullsized":"2024-09-21T14:22:16.938521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.858853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":56085909,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":5683,"num_read_kb":27782,"num_write":542,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"71'212","reported_seq":1501,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.982149+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:22:16.982149+0000","last_peered":"2024-09-21T14:22:16.982149+0000","last_clean":"2024-09-21T14:22:16.982149+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:22:16.982149+0000","last_undegraded":"2024-09-21T14:22:16.982149+0000","last_fullsized":"2024-09-21T14:22:16.982149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":27903568,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":1373,"num_read_kb":10053,"num_write":203,"num_write_kb":7650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"71'623","reported_seq":2084,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.089414+0000","last_change":"2024-09-21T14:21:20.149882+0000","last_active":"2024-09-21T14:22:17.089414+0000","last_peered":"2024-09-21T14:22:17.089414+0000","last_clean":"2024-09-21T14:22:17.089414+0000","last_became_active":"2024-09-21T14:21:18.148649+0000","last_became_peered":"2024-09-21T14:21:18.148649+0000","last_unstale":"2024-09-21T14:22:17.089414+0000","last_undegraded":"2024-09-21T14:22:17.089414+0000","last_fullsized":"2024-09-21T14:22:17.089414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:21.298648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":17769067,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":4243,"num_read_kb":8642,"num_write":839,"num_write_kb":4171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"71'392","reported_seq":2041,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.583799+0000","last_change":"2024-09-21T14:21:52.139246+0000","last_active":"2024-09-21T14:22:16.583799+0000","last_peered":"2024-09-21T14:22:16.583799+0000","last_clean":"2024-09-21T14:22:16.583799+0000","last_became_active":"2024-09-21T14:21:52.138595+0000","last_became_peered":"2024-09-21T14:21:52.138595+0000","last_unstale":"2024-09-21T14:22:16.583799+0000","last_undegraded":"2024-09-21T14:22:16.583799+0000","last_fullsized":"2024-09-21T14:22:16.583799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:23.662981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":19491232,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":2877,"num_read_kb":11545,"num_write":532,"num_write_kb":5961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":83845,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"71'468","reported_seq":2027,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.963003+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:22:16.963003+0000","last_peered":"2024-09-21T14:22:16.963003+0000","last_clean":"2024-09-21T14:22:16.963003+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:22:16.963003+0000","last_undegraded":"2024-09-21T14:22:16.963003+0000","last_fullsized":"2024-09-21T14:22:16.963003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":38757879,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3403,"num_read_kb":22904,"num_write":613,"num_write_kb":8914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"71'337","reported_seq":2389,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.496211+0000","last_change":"2024-09-21T14:21:52.140205+0000","last_active":"2024-09-21T14:22:16.496211+0000","last_peered":"2024-09-21T14:22:16.496211+0000","last_clean":"2024-09-21T14:22:16.496211+0000","last_became_active":"2024-09-21T14:21:52.138972+0000","last_became_peered":"2024-09-21T14:21:52.138972+0000","last_unstale":"2024-09-21T14:22:16.496211+0000","last_undegraded":"2024-09-21T14:22:16.496211+0000","last_fullsized":"2024-09-21T14:22:16.496211+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:41.279903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":34367430,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":28,"num_read":3887,"num_read_kb":15677,"num_write":406,"num_write_kb":9782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1075708,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.491011+0000","last_change":"2024-09-21T14:21:10.671401+0000","last_active":"2024-09-21T14:22:14.491011+0000","last_peered":"2024-09-21T14:22:14.491011+0000","last_clean":"2024-09-21T14:22:14.491011+0000","last_became_active":"2024-09-21T14:21:08.709241+0000","last_became_peered":"2024-09-21T14:21:08.709241+0000","last_unstale":"2024-09-21T14:22:14.491011+0000","last_undegraded":"2024-09-21T14:22:14.491011+0000","last_fullsized":"2024-09-21T14:22:14.491011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:30.375706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033166100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'309","reported_seq":2227,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.058292+0000","last_change":"2024-09-21T14:21:52.138940+0000","last_active":"2024-09-21T14:22:17.058292+0000","last_peered":"2024-09-21T14:22:17.058292+0000","last_clean":"2024-09-21T14:22:17.058292+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:22:17.058292+0000","last_undegraded":"2024-09-21T14:22:17.058292+0000","last_fullsized":"2024-09-21T14:22:17.058292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":48180375,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":24,"num_read":2585,"num_read_kb":9914,"num_write":316,"num_write_kb":5547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.483714+0000","last_change":"2024-09-21T14:21:10.745650+0000","last_active":"2024-09-21T14:22:14.483714+0000","last_peered":"2024-09-21T14:22:14.483714+0000","last_clean":"2024-09-21T14:22:14.483714+0000","last_became_active":"2024-09-21T14:21:08.709869+0000","last_became_peered":"2024-09-21T14:21:08.709869+0000","last_unstale":"2024-09-21T14:22:14.483714+0000","last_undegraded":"2024-09-21T14:22:14.483714+0000","last_fullsized":"2024-09-21T14:22:14.483714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:08.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031137599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"71'403","reported_seq":2323,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.130879+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:22:17.130879+0000","last_peered":"2024-09-21T14:22:17.130879+0000","last_clean":"2024-09-21T14:22:17.130879+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:22:17.130879+0000","last_undegraded":"2024-09-21T14:22:17.130879+0000","last_fullsized":"2024-09-21T14:22:17.130879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":33305860,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":3302,"num_read_kb":10872,"num_write":539,"num_write_kb":4266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.541897+0000","last_change":"2024-09-21T14:21:49.338538+0000","last_active":"2024-09-21T14:22:14.541897+0000","last_peered":"2024-09-21T14:22:14.541897+0000","last_clean":"2024-09-21T14:22:14.541897+0000","last_became_active":"2024-09-21T14:21:49.338272+0000","last_became_peered":"2024-09-21T14:21:49.338272+0000","last_unstale":"2024-09-21T14:22:14.541897+0000","last_undegraded":"2024-09-21T14:22:14.541897+0000","last_fullsized":"2024-09-21T14:22:14.541897+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:55.449259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"71'271","reported_seq":2379,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.911707+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:22:16.911707+0000","last_peered":"2024-09-21T14:22:16.911707+0000","last_clean":"2024-09-21T14:22:16.911707+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:22:16.911707+0000","last_undegraded":"2024-09-21T14:22:16.911707+0000","last_fullsized":"2024-09-21T14:22:16.911707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":43451172,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":2286,"num_read_kb":30676,"num_write":260,"num_write_kb":8647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.510036+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:22:14.510036+0000","last_peered":"2024-09-21T14:22:14.510036+0000","last_clean":"2024-09-21T14:22:14.510036+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:22:14.510036+0000","last_undegraded":"2024-09-21T14:22:14.510036+0000","last_fullsized":"2024-09-21T14:22:14.510036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"71'323","reported_seq":4809,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.130656+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:22:17.130656+0000","last_peered":"2024-09-21T14:22:17.130656+0000","last_clean":"2024-09-21T14:22:17.130656+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:22:17.130656+0000","last_undegraded":"2024-09-21T14:22:17.130656+0000","last_fullsized":"2024-09-21T14:22:17.130656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":53138694,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":5718,"num_read_kb":29498,"num_write":306,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"71'237","reported_seq":2274,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.980746+0000","last_change":"2024-09-21T14:21:52.139305+0000","last_active":"2024-09-21T14:22:16.980746+0000","last_peered":"2024-09-21T14:22:16.980746+0000","last_clean":"2024-09-21T14:22:16.980746+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:22:16.980746+0000","last_undegraded":"2024-09-21T14:22:16.980746+0000","last_fullsized":"2024-09-21T14:22:16.980746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":37291064,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":2967,"num_read_kb":20527,"num_write":221,"num_write_kb":8115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":214,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.483374+0000","last_change":"2024-09-21T14:21:09.739244+0000","last_active":"2024-09-21T14:22:14.483374+0000","last_peered":"2024-09-21T14:22:14.483374+0000","last_clean":"2024-09-21T14:22:14.483374+0000","last_became_active":"2024-09-21T14:21:09.738135+0000","last_became_peered":"2024-09-21T14:21:09.738135+0000","last_unstale":"2024-09-21T14:22:14.483374+0000","last_undegraded":"2024-09-21T14:22:14.483374+0000","last_fullsized":"2024-09-21T14:22:14.483374+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":71,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.487255+0000","last_change":"2024-09-21T14:21:10.669229+0000","last_active":"2024-09-21T14:22:14.487255+0000","last_peered":"2024-09-21T14:22:14.487255+0000","last_clean":"2024-09-21T14:22:14.487255+0000","last_became_active":"2024-09-21T14:21:08.536063+0000","last_became_peered":"2024-09-21T14:21:08.536063+0000","last_unstale":"2024-09-21T14:22:14.487255+0000","last_undegraded":"2024-09-21T14:22:14.487255+0000","last_fullsized":"2024-09-21T14:22:14.487255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:50.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"71'587","reported_seq":2237,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.644087+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:22:16.644087+0000","last_peered":"2024-09-21T14:22:16.644087+0000","last_clean":"2024-09-21T14:22:16.644087+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:22:16.644087+0000","last_undegraded":"2024-09-21T14:22:16.644087+0000","last_fullsized":"2024-09-21T14:22:16.644087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":26730633,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":3892,"num_read_kb":10252,"num_write":827,"num_write_kb":6268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":72,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.484749+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:22:14.484749+0000","last_peered":"2024-09-21T14:22:14.484749+0000","last_clean":"2024-09-21T14:22:14.484749+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:22:14.484749+0000","last_undegraded":"2024-09-21T14:22:14.484749+0000","last_fullsized":"2024-09-21T14:22:14.484749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"71'835","reported_seq":1678,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.130744+0000","last_change":"2024-09-21T14:21:52.139498+0000","last_active":"2024-09-21T14:22:17.130744+0000","last_peered":"2024-09-21T14:22:17.130744+0000","last_clean":"2024-09-21T14:22:17.130744+0000","last_became_active":"2024-09-21T14:21:52.138744+0000","last_became_peered":"2024-09-21T14:21:52.138744+0000","last_unstale":"2024-09-21T14:22:17.130744+0000","last_undegraded":"2024-09-21T14:22:17.130744+0000","last_fullsized":"2024-09-21T14:22:17.130744+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:16:08.802429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":30932068,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":19,"num_read":3696,"num_read_kb":12924,"num_write":1255,"num_write_kb":7265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2088730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.523837+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:22:14.523837+0000","last_peered":"2024-09-21T14:22:14.523837+0000","last_clean":"2024-09-21T14:22:14.523837+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:22:14.523837+0000","last_undegraded":"2024-09-21T14:22:14.523837+0000","last_fullsized":"2024-09-21T14:22:14.523837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"70'453","reported_seq":1808,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:15.461512+0000","last_change":"2024-09-21T14:21:52.139320+0000","last_active":"2024-09-21T14:22:15.461512+0000","last_peered":"2024-09-21T14:22:15.461512+0000","last_clean":"2024-09-21T14:22:15.461512+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:22:15.461512+0000","last_undegraded":"2024-09-21T14:22:15.461512+0000","last_fullsized":"2024-09-21T14:22:15.461512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":62763701,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":2394,"num_read_kb":9621,"num_write":587,"num_write_kb":5237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T14:22:14.490838+0000","last_change":"2024-09-21T14:21:10.671402+0000","last_active":"2024-09-21T14:22:14.490838+0000","last_peered":"2024-09-21T14:22:14.490838+0000","last_clean":"2024-09-21T14:22:14.490838+0000","last_became_active":"2024-09-21T14:21:08.536435+0000","last_became_peered":"2024-09-21T14:21:08.536435+0000","last_unstale":"2024-09-21T14:22:14.490838+0000","last_undegraded":"2024-09-21T14:22:14.490838+0000","last_fullsized":"2024-09-21T14:22:14.490838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.665955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032570599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"71'515","reported_seq":3046,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.093748+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:22:17.093748+0000","last_peered":"2024-09-21T14:22:17.093748+0000","last_clean":"2024-09-21T14:22:17.093748+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:22:17.093748+0000","last_undegraded":"2024-09-21T14:22:17.093748+0000","last_fullsized":"2024-09-21T14:22:17.093748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":79982163,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":5287,"num_read_kb":13818,"num_write":619,"num_write_kb":12186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"71'511","reported_seq":3365,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.890071+0000","last_change":"2024-09-21T14:21:20.161344+0000","last_active":"2024-09-21T14:22:16.890071+0000","last_peered":"2024-09-21T14:22:16.890071+0000","last_clean":"2024-09-21T14:22:16.890071+0000","last_became_active":"2024-09-21T14:21:18.147766+0000","last_became_peered":"2024-09-21T14:21:18.147766+0000","last_unstale":"2024-09-21T14:22:16.890071+0000","last_undegraded":"2024-09-21T14:22:16.890071+0000","last_fullsized":"2024-09-21T14:22:16.890071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:19:19.787328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":74392810,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":6411,"num_read_kb":17623,"num_write":498,"num_write_kb":6014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"71'292","reported_seq":822,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.953602+0000","last_change":"2024-09-21T14:21:20.161437+0000","last_active":"2024-09-21T14:22:16.953602+0000","last_peered":"2024-09-21T14:22:16.953602+0000","last_clean":"2024-09-21T14:22:16.953602+0000","last_became_active":"2024-09-21T14:21:18.143098+0000","last_became_peered":"2024-09-21T14:21:18.143098+0000","last_unstale":"2024-09-21T14:22:16.953602+0000","last_undegraded":"2024-09-21T14:22:16.953602+0000","last_fullsized":"2024-09-21T14:22:16.953602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:16.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":19078137,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":22,"num_read":827,"num_read_kb":6591,"num_write":301,"num_write_kb":6953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"71'334","reported_seq":1747,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.137978+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:22:17.137978+0000","last_peered":"2024-09-21T14:22:17.137978+0000","last_clean":"2024-09-21T14:22:17.137978+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:22:17.137978+0000","last_undegraded":"2024-09-21T14:22:17.137978+0000","last_fullsized":"2024-09-21T14:22:17.137978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":37485203,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":2029,"num_read_kb":6895,"num_write":468,"num_write_kb":4837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"71'597","reported_seq":1747,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.130804+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:22:17.130804+0000","last_peered":"2024-09-21T14:22:17.130804+0000","last_clean":"2024-09-21T14:22:17.130804+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:22:17.130804+0000","last_undegraded":"2024-09-21T14:22:17.130804+0000","last_fullsized":"2024-09-21T14:22:17.130804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":46045739,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":22,"num_read":3281,"num_read_kb":13830,"num_write":808,"num_write_kb":8544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"71'242","reported_seq":2026,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.101988+0000","last_change":"2024-09-21T14:21:34.229265+0000","last_active":"2024-09-21T14:22:17.101988+0000","last_peered":"2024-09-21T14:22:17.101988+0000","last_clean":"2024-09-21T14:22:17.101988+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:22:17.101988+0000","last_undegraded":"2024-09-21T14:22:17.101988+0000","last_fullsized":"2024-09-21T14:22:17.101988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:04.756321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":26232771,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2089,"num_read_kb":12686,"num_write":228,"num_write_kb":6104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"71'866","reported_seq":3184,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.131208+0000","last_change":"2024-09-21T14:21:52.141637+0000","last_active":"2024-09-21T14:22:17.131208+0000","last_peered":"2024-09-21T14:22:17.131208+0000","last_clean":"2024-09-21T14:22:17.131208+0000","last_became_active":"2024-09-21T14:21:52.140890+0000","last_became_peered":"2024-09-21T14:21:52.140890+0000","last_unstale":"2024-09-21T14:22:17.131208+0000","last_undegraded":"2024-09-21T14:22:17.131208+0000","last_fullsized":"2024-09-21T14:22:17.131208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:12.353305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":64809317,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":5685,"num_read_kb":17901,"num_write":1345,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":3052424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"71'313","reported_seq":3180,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.131010+0000","last_change":"2024-09-21T14:21:20.151725+0000","last_active":"2024-09-21T14:22:17.131010+0000","last_peered":"2024-09-21T14:22:17.131010+0000","last_clean":"2024-09-21T14:22:17.131010+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:22:17.131010+0000","last_undegraded":"2024-09-21T14:22:17.131010+0000","last_fullsized":"2024-09-21T14:22:17.131010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:56.018570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":70045597,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":22,"num_read":3244,"num_read_kb":20027,"num_write":306,"num_write_kb":20297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"71'304","reported_seq":2310,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.911746+0000","last_change":"2024-09-21T14:21:20.143653+0000","last_active":"2024-09-21T14:22:16.911746+0000","last_peered":"2024-09-21T14:22:16.911746+0000","last_clean":"2024-09-21T14:22:16.911746+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:22:16.911746+0000","last_undegraded":"2024-09-21T14:22:16.911746+0000","last_fullsized":"2024-09-21T14:22:16.911746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:14.454034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":44902377,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":2513,"num_read_kb":14035,"num_write":296,"num_write_kb":5305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"71'247","reported_seq":1626,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.942720+0000","last_change":"2024-09-21T14:21:52.139083+0000","last_active":"2024-09-21T14:22:16.942720+0000","last_peered":"2024-09-21T14:22:16.942720+0000","last_clean":"2024-09-21T14:22:16.942720+0000","last_became_active":"2024-09-21T14:21:52.138188+0000","last_became_peered":"2024-09-21T14:21:52.138188+0000","last_unstale":"2024-09-21T14:22:16.942720+0000","last_undegraded":"2024-09-21T14:22:16.942720+0000","last_fullsized":"2024-09-21T14:22:16.942720+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:15.976782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":56133083,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":1604,"num_read_kb":20459,"num_write":308,"num_write_kb":17665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":3452172,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"71'352","reported_seq":2252,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:17.133301+0000","last_change":"2024-09-21T14:21:20.756124+0000","last_active":"2024-09-21T14:22:17.133301+0000","last_peered":"2024-09-21T14:22:17.133301+0000","last_clean":"2024-09-21T14:22:17.133301+0000","last_became_active":"2024-09-21T14:21:18.147754+0000","last_became_peered":"2024-09-21T14:21:18.147754+0000","last_unstale":"2024-09-21T14:22:17.133301+0000","last_undegraded":"2024-09-21T14:22:17.133301+0000","last_fullsized":"2024-09-21T14:22:17.133301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:20.375487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":33740785,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":3376,"num_read_kb":20057,"num_write":414,"num_write_kb":10594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"70'886","reported_seq":3293,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T14:22:16.496171+0000","last_change":"2024-09-21T14:21:20.161546+0000","last_active":"2024-09-21T14:22:16.496171+0000","last_peered":"2024-09-21T14:22:16.496171+0000","last_clean":"2024-09-21T14:22:16.496171+0000","last_became_active":"2024-09-21T14:21:18.146217+0000","last_became_peered":"2024-09-21T14:21:18.146217+0000","last_unstale":"2024-09-21T14:22:16.496171+0000","last_undegraded":"2024-09-21T14:22:16.496171+0000","last_fullsized":"2024-09-21T14:22:16.496171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:34.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":25106998,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":6736,"num_read_kb":21100,"num_write":1302,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1358757820,"num_objects":1703,"num_object_clones":461,"num_object_copies":3406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1703,"num_whiteouts":682,"num_read":112320,"num_read_kb":535490,"num_write":16195,"num_write_kb":262939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":16773182,"num_keys_recovered":58,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":46379008,"data_stored":395425353,"data_compressed":24894396,"data_compressed_allocated":27111424,"data_compressed_original":379501588,"omap_allocated":0,"internal_metadata":0},"log_size":13191,"ondisk_log_size":13191,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64820,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38832,"kb_used_data":3620,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333008,"statfs":{"total":96636764160,"available":96597000192,"internally_reserved":0,"allocated":3706880,"data_stored":24662281,"data_compressed":1417578,"data_compressed_allocated":1609728,"data_compressed_original":23323825,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.20899999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.307},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.373}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.248},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.318}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.33600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40996,"kb_used_data":5788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330844,"statfs":{"total":96636764160,"available":96594784256,"internally_reserved":0,"allocated":5926912,"data_stored":46382036,"data_compressed":2895824,"data_compressed_allocated":3133440,"data_compressed_original":44405513,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.24399999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.32600000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.374}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35799999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.68600000000000005}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.435}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45032,"kb_used_data":5728,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326808,"statfs":{"total":96636764160,"available":96590651392,"internally_reserved":0,"allocated":5865472,"data_stored":37237037,"data_compressed":2353545,"data_compressed_allocated":2801664,"data_compressed_original":35059258,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.442}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.312},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.7430000000000003,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.28999999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.36199999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35720,"kb_used_data":8704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336120,"statfs":{"total":96636764160,"available":96600186880,"internally_reserved":0,"allocated":8912896,"data_stored":70282024,"data_compressed":4534963,"data_compressed_allocated":4714496,"data_compressed_original":67195763,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.33400000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.312},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.36199999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.374},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.439}]},{"osd":7,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.39600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":8728576,"data_stored":68977141,"data_compressed":4301846,"data_compressed_allocated":4714496,"data_compressed_original":65725593,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.316},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38}]},{"osd":4,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.72899999999999998}]}]},{"osd":2,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46384,"kb_used_data":7076,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325456,"statfs":{"total":96636764160,"available":96589266944,"internally_reserved":0,"allocated":7245824,"data_stored":53891567,"data_compressed":3439091,"data_compressed_allocated":3874816,"data_compressed_original":51384961,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.36199999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.29999999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.38800000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.44900000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41899999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.436}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76444,"kb_used_data":9236,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94295396,"statfs":{"total":96636764160,"available":96558485504,"internally_reserved":0,"allocated":9457664,"data_stored":81136110,"data_compressed":5094884,"data_compressed_allocated":5136384,"data_compressed_original":77865962,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.219},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28199999999999997}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.23499999999999999}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.39200000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.33200000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.318},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.376}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.41799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8626176,"data_stored":80757430,"data_compressed":5094884,"data_compressed_allocated":5136384,"data_compressed_original":77865962,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6541312,"data_stored":53653076,"data_compressed":3441573,"data_compressed_allocated":3883008,"data_compressed_original":51434113,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7987200,"data_stored":68184736,"data_compressed":4271825,"data_compressed_allocated":4673536,"data_compressed_original":65201305,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8151040,"data_stored":70117914,"data_compressed":4565776,"data_compressed_allocated":4714496,"data_compressed_original":67353345,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4972544,"data_stored":36265118,"data_compressed":2321135,"data_compressed_allocated":2756608,"data_compressed_original":34485818,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5177344,"data_stored":46043605,"data_compressed":2893459,"data_compressed_allocated":3125248,"data_compressed_original":44356361,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2924544,"data_stored":24328759,"data_compressed":1417578,"data_compressed_allocated":1609728,"data_compressed_original":23323825,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:22:18.348 INFO:tasks.thrashosds.thrasher:Setting 3.e to [4, 7] 2024-09-21T14:22:18.348 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.e', '4', '7'] 2024-09-21T14:22:18.348 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.e 4 7 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5403 writesame 0xbc5345b thru 0xbc571e6 data_size 0x6(0x3d8c bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5406 punch from 0x427a664 to 0x428229b, (0x7c37 bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5408 punch from 0x7cbd475 to 0x7cccec5, (0xfa50 bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5410 read 0xa3c79d4 thru 0xa3d4595 (0xcbc2 bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5411 read 0x1eacd17 thru 0x1eb3860 (0x6b4a bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5413 read 0x493deae thru 0x4947548 (0x969b bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5414 read 0x343452b thru 0x343a8f4 (0x63ca bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:5415 read 0x162d77 thru 0x16a249 (0x74d3 bytes) 2024-09-21T14:22:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.351 INFO:teuthology.orchestra.run.smithi073.stdout:5417 read 0xb69c60 thru 0xb6e501 (0x48a2 bytes) 2024-09-21T14:22:18.351 INFO:teuthology.orchestra.run.smithi073.stdout:5419 write 0x484e3a1 thru 0x485b77d (0xd3dd bytes) 2024-09-21T14:22:18.355 INFO:teuthology.orchestra.run.smithi073.stdout:5420 writesame 0x3d69a00 thru 0x3d78655 data_size 0x1a(0xec56 bytes) 2024-09-21T14:22:18.360 INFO:teuthology.orchestra.run.smithi073.stdout:5421 punch from 0x10a5cb2 to 0x10a6cfe, (0x104c bytes) 2024-09-21T14:22:18.422 INFO:teuthology.orchestra.run.smithi073.stdout:5422 write 0xa236113 thru 0xa23cb08 (0x69f6 bytes) 2024-09-21T14:22:18.424 INFO:teuthology.orchestra.run.smithi073.stdout:5423 read 0x2cbe3d3 thru 0x2ccdc91 (0xf8bf bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5424 write 0x202b4f5 thru 0x2033369 (0x7e75 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5427 read 0x83cb634 thru 0x83d69d1 (0xb39e bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5428 writesame 0x2832094 thru 0x283d513 data_size 0x80(0xb480 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5430 write 0x6b53d87 thru 0x6b60e5a (0xd0d4 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5431 write 0xe0687c6 thru 0xe073e9e (0xb6d9 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5432 writesame 0x9fe41d3 thru 0x9fe72e1 data_size 0x1(0x310f bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5434 write 0x99c5752 thru 0x99cb621 (0x5ed0 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5436 read 0xb745941 thru 0xb745997 (0x57 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5437 punch from 0x6924d15 to 0x6928b17, (0x3e02 bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5438 read 0xc4c0c4 thru 0xc559de (0x991b bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5439 writesame 0xdba31da thru 0xdbaf038 data_size 0x13(0xbe5f bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5440 writesame 0xdda8c51 thru 0xdda9bae data_size 0x7(0xf5e bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5441 write 0xa3c6ef thru 0xa42efb (0x680d bytes) 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.425 INFO:teuthology.orchestra.run.smithi073.stdout:5443 read 0xcdd5b65 thru 0xcdd87a8 (0x2c44 bytes) 2024-09-21T14:22:18.426 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.426 INFO:teuthology.orchestra.run.smithi073.stdout:5447 write 0x26d4b9f thru 0x26e2b3a (0xdf9c bytes) 2024-09-21T14:22:18.426 INFO:teuthology.orchestra.run.smithi073.stdout:5449 punch from 0xd7ca259 to 0xd7d9c99, (0xfa40 bytes) 2024-09-21T14:22:18.426 INFO:teuthology.orchestra.run.smithi073.stdout:5450 read 0x1253fc5 thru 0x125e75e (0xa79a bytes) 2024-09-21T14:22:18.426 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:18.424+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:18.426 INFO:teuthology.orchestra.run.smithi073.stdout:5452 write 0xed5d25a thru 0xed60919 (0x36c0 bytes) 2024-09-21T14:22:18.429 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:18.429 INFO:teuthology.orchestra.run.smithi073.stdout:5455 read 0xd3888cc thru 0xd390ca2 (0x83d7 bytes) 2024-09-21T14:22:18.429 INFO:teuthology.orchestra.run.smithi073.stdout:5456 writesame 0x4c257fa thru 0x4c2b6f9 data_size 0x5f(0x5f00 bytes) 2024-09-21T14:22:18.435 INFO:teuthology.orchestra.run.smithi073.stdout:5460 read 0xa388147 thru 0xa3895b6 (0x1470 bytes) 2024-09-21T14:22:18.435 INFO:teuthology.orchestra.run.smithi073.stdout:5462 write 0x2b6528c thru 0x2b69de3 (0x4b58 bytes) 2024-09-21T14:22:18.442 INFO:teuthology.orchestra.run.smithi073.stdout:5463 read 0x462f8e7 thru 0x4635be1 (0x62fb bytes) 2024-09-21T14:22:18.442 INFO:teuthology.orchestra.run.smithi073.stdout:5464 write 0xbf3d2b0 thru 0xbf4100d (0x3d5e bytes) 2024-09-21T14:22:18.445 INFO:teuthology.orchestra.run.smithi073.stdout:5466 write 0x87ca307 thru 0x87cdfab (0x3ca5 bytes) 2024-09-21T14:22:18.449 INFO:teuthology.orchestra.run.smithi073.stdout:5467 write 0x210fdeb thru 0x211b594 (0xb7aa bytes) 2024-09-21T14:22:18.450 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.450 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.457 INFO:teuthology.orchestra.run.smithi073.stdout:5472 write 0xc6b54ec thru 0xc6c0817 (0xb32c bytes) 2024-09-21T14:22:18.460 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:18.469 INFO:teuthology.orchestra.run.smithi073.stdout:5474 write 0x891399e thru 0x891579b (0x1dfe bytes) 2024-09-21T14:22:18.476 INFO:teuthology.orchestra.run.smithi073.stdout:5475 write 0x68f9677 thru 0x68fb39f (0x1d29 bytes) 2024-09-21T14:22:18.480 INFO:teuthology.orchestra.run.smithi073.stdout:5476 read 0x3127e7 thru 0x3140c7 (0x18e1 bytes) 2024-09-21T14:22:18.480 INFO:teuthology.orchestra.run.smithi073.stdout:5477 write 0xa75b99c thru 0xa762fe5 (0x764a bytes) 2024-09-21T14:22:18.484 INFO:teuthology.orchestra.run.smithi073.stdout:5479 read 0xcc48718 thru 0xcc49e43 (0x172c bytes) 2024-09-21T14:22:18.484 INFO:teuthology.orchestra.run.smithi073.stdout:5480 punch from 0x64f7e4a to 0x6504f6b, (0xd121 bytes) 2024-09-21T14:22:18.496 INFO:teuthology.orchestra.run.smithi073.stdout:5481 trunc from 0xed6091a to 0x5229495 2024-09-21T14:22:18.522 INFO:teuthology.orchestra.run.smithi073.stdout:5482 punch from 0x3d9cfd4 to 0x3da6bcb, (0x9bf7 bytes) 2024-09-21T14:22:18.525 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:18.524+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:18.533 INFO:teuthology.orchestra.run.smithi073.stdout:5483 write 0x82c2c11 thru 0x82c9fb3 (0x73a3 bytes) 2024-09-21T14:22:18.540 INFO:teuthology.orchestra.run.smithi073.stdout:5484 trunc from 0x82c9fb4 to 0x8d09bf3 2024-09-21T14:22:18.545 INFO:teuthology.orchestra.run.smithi073.stdout:5485 read 0x1d9dd99 thru 0x1da0527 (0x278f bytes) 2024-09-21T14:22:18.551 INFO:teuthology.orchestra.run.smithi073.stdout:5486 trunc from 0x8d09bf3 to 0xdc0a8b2 2024-09-21T14:22:18.557 INFO:teuthology.orchestra.run.smithi073.stdout:5487 read 0xa437bb5 thru 0xa43d2aa (0x56f6 bytes) 2024-09-21T14:22:18.557 INFO:teuthology.orchestra.run.smithi073.stdout:5489 write 0x1221886 thru 0x1228222 (0x699d bytes) 2024-09-21T14:22:18.564 INFO:teuthology.orchestra.run.smithi073.stdout:5490 punch from 0x22c13e3 to 0x22cac32, (0x984f bytes) 2024-09-21T14:22:18.564 INFO:teuthology.orchestra.run.smithi073.stdout:5494 write 0x87a1b76 thru 0x87a5284 (0x370f bytes) 2024-09-21T14:22:18.569 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.569 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.571 INFO:teuthology.orchestra.run.smithi073.stdout:5496 write 0x1fe9394 thru 0x1ff1651 (0x82be bytes) 2024-09-21T14:22:18.573 INFO:teuthology.orchestra.run.smithi073.stdout:5497 read 0x13697b thru 0x13ce20 (0x64a6 bytes) 2024-09-21T14:22:18.574 INFO:teuthology.orchestra.run.smithi073.stdout:5498 punch from 0x17a6e14 to 0x17aac7b, (0x3e67 bytes) 2024-09-21T14:22:18.574 INFO:teuthology.orchestra.run.smithi073.stdout:5499 write 0xa749135 thru 0xa74f41f (0x62eb bytes) 2024-09-21T14:22:18.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:18.579 INFO:teuthology.orchestra.run.smithi073.stdout:5500 punch from 0x30f2bb8 to 0x30f40df, (0x1527 bytes) 2024-09-21T14:22:18.585 INFO:teuthology.orchestra.run.smithi073.stdout:5501 writesame 0xe59bfd2 thru 0xe5aa4cd data_size 0x1e(0xe4fc bytes) 2024-09-21T14:22:18.585 INFO:teuthology.orchestra.run.smithi073.stdout:5502 write 0xa3db75b thru 0xa3df350 (0x3bf6 bytes) 2024-09-21T14:22:18.589 INFO:teuthology.orchestra.run.smithi073.stdout:5503 write 0x65f23a7 thru 0x65fe6c0 (0xc31a bytes) 2024-09-21T14:22:18.593 INFO:teuthology.orchestra.run.smithi073.stdout:5507 punch from 0xd7d1593 to 0xd7d8906, (0x7373 bytes) 2024-09-21T14:22:18.600 INFO:teuthology.orchestra.run.smithi073.stdout:5508 trunc from 0xe5aa4ce to 0x86b8008 2024-09-21T14:22:18.610 INFO:teuthology.orchestra.run.smithi073.stdout:5509 read 0x2a7549b thru 0x2a7c2bf (0x6e25 bytes) 2024-09-21T14:22:18.610 INFO:teuthology.orchestra.run.smithi073.stdout:5511 read 0x34ff2f4 thru 0x3507568 (0x8275 bytes) 2024-09-21T14:22:18.610 INFO:teuthology.orchestra.run.smithi073.stdout:5512 write 0x28480ee thru 0x2848cd9 (0xbec bytes) 2024-09-21T14:22:18.611 INFO:teuthology.orchestra.run.smithi073.stdout:5513 write 0x4365cfd thru 0x43677a4 (0x1aa8 bytes) 2024-09-21T14:22:18.626 INFO:teuthology.orchestra.run.smithi073.stdout:5514 writesame 0xd0cb3ca thru 0xd0d1277 data_size 0x1(0x5eae bytes) 2024-09-21T14:22:18.626 INFO:teuthology.orchestra.run.smithi073.stdout:5515 punch from 0xca1a55f to 0xca1ce29, (0x28ca bytes) 2024-09-21T14:22:18.626 INFO:teuthology.orchestra.run.smithi073.stdout:5516 read 0xc2930a1 thru 0xc2940ec (0x104c bytes) 2024-09-21T14:22:18.626 INFO:teuthology.orchestra.run.smithi073.stdout:5517 write 0x92b3a60 thru 0x92c0a23 (0xcfc4 bytes) 2024-09-21T14:22:18.631 INFO:teuthology.orchestra.run.smithi073.stdout:5518 write 0x4fd246b thru 0x4fd25c3 (0x159 bytes) 2024-09-21T14:22:18.634 INFO:teuthology.orchestra.run.smithi073.stdout:5519 write 0xb89d31b thru 0xb8a62ea (0x8fd0 bytes) 2024-09-21T14:22:18.643 INFO:teuthology.orchestra.run.smithi073.stdout:5521 writesame 0xd93495a thru 0xd93bb59 data_size 0x10(0x7200 bytes) 2024-09-21T14:22:18.643 INFO:teuthology.orchestra.run.smithi073.stdout:5523 clone 40 order 21 su 65536 sc 3 2024-09-21T14:22:18.663 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:18.662+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:18.723 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.723 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.726 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:18.726+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:18.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:18.827 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:18.827+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:18.874 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.874 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:18.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:18.928 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:18.927+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:18.984 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:18.985 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:18.993 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:19.027 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:19.028 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:19.028 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:19.029+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:19.044 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:19.044 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:19.059 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:19.059 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:19.074 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:19.075 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:19.090 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:19.107 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:19.122 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:19.123 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:19.130 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:19.130+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:19.135 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:19.135 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:19.138 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:19.139 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:19.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:19.230 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:19.230+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:19.248 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.248 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:19.248 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:19.248 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:19.256 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:19.332 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:19.332+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:19.400 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:19.400 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:19.400 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:19.400 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:19.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:19.453 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:19.453+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:19.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:22:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:set 3.e pg_upmap mapping to [4,7] 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43037 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130267955999997, 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044045162999999998, 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43037 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:19.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382702+0000", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382704+0000", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1880000000000001e-06 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382710+0000", 2024-09-21T14:22:19.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.507e-06 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382714+0000", 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2030000000000002e-06 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426582+0000", 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043868468000000001 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426657+0000", 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4577000000000001e-05 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426748+0000", 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.056e-05 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43038 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 319488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:19.518 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130252073000001, 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044219614999999997, 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43038 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382718+0000", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382724+0000", 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5659999999999997e-06 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382729+0000", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382733+0000", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426770+0000", 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044036868 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426847+0000", 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7068000000000001e-05 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426938+0000", 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0708e-05 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.522 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43039 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130205959000001, 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044367363999999999, 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43039 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382764+0000", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382767+0000", 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5959999999999998e-06 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382773+0000", 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6710000000000002e-06 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382779+0000", 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2970000000000003e-06 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.426961+0000", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044182402000000003 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427041+0000", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9810999999999999e-05 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427132+0000", 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1025000000000003e-05 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43040 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 344064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130187712000001, 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044540541000000003, 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43040 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382783+0000", 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382782+0000", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382784+0000", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0209999999999998e-06 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382790+0000", 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5130000000000004e-06 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382794+0000", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2200000000000003e-06 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427155+0000", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044360978000000002 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427234+0000", 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8580999999999999e-05 2024-09-21T14:22:19.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427323+0000", 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9584000000000002e-05 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43041 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 354304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130172449, 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044711682000000003, 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.531 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43041 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382798+0000", 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382799+0000", 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382805+0000", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7259999999999999e-06 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382810+0000", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2910000000000001e-06 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427346+0000", 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044536282000000003 2024-09-21T14:22:19.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427420+0000", 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4651000000000001e-05 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427510+0000", 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9147999999999994e-05 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43042 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 364544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130155522000003, 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044890194000000001, 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43042 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382815+0000", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382814+0000", 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382816+0000", 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9259999999999999e-06 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382822+0000", 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.733e-06 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382826+0000", 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.929e-06 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427536+0000", 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044709567999999998 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427613+0000", 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7390000000000003e-05 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427705+0000", 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1984000000000006e-05 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43043 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130140482999998, 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045064142000000001, 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.540 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43043 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382830+0000", 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382831+0000", 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.874e-06 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382837+0000", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8320000000000002e-06 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382841+0000", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1500000000000001e-06 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427728+0000", 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044886479 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427805+0000", 2024-09-21T14:22:19.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.661e-05 2024-09-21T14:22:19.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427894+0000", 2024-09-21T14:22:19.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9488999999999996e-05 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43044 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130125130000003, 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045458112000000002, 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.546 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43044 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382845+0000", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382847+0000", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9810000000000002e-06 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382853+0000", 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7880000000000003e-06 2024-09-21T14:22:19.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382857+0000", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9790000000000004e-06 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427917+0000", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045060351999999998 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428174+0000", 2024-09-21T14:22:19.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025663200000000002 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428303+0000", 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129701 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43045 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:19.550 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130109779999998, 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045669176999999998, 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43045 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382861+0000", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382860+0000", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382862+0000", 2024-09-21T14:22:19.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382868+0000", 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6500000000000001e-06 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382873+0000", 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5570000000000003e-06 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428330+0000", 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045456929 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428406+0000", 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5184000000000004e-05 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428530+0000", 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124213 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43046 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130093143000003, 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873328999999997, 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43046 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.555 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382877+0000", 2024-09-21T14:22:19.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382879+0000", 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9800000000000001e-06 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382884+0000", 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5929999999999996e-06 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382888+0000", 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9249999999999997e-06 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428559+0000", 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045670193999999997 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428632+0000", 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2978999999999994e-05 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428751+0000", 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119015 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43047 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130077966000002, 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046080838999999998, 2024-09-21T14:22:19.559 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43047 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382892+0000", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382894+0000", 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.956e-06 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382900+0000", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.541e-06 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382904+0000", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0230000000000003e-06 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428777+0000", 2024-09-21T14:22:19.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045873239000000003 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428850+0000", 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2762000000000006e-05 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428973+0000", 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012365400000000001 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43048 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130063026999998, 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046302055000000002, 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43048 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382907+0000", 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382909+0000", 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.835e-06 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382927+0000", 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8580000000000002e-05 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382932+0000", 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5419999999999999e-06 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429004+0000", 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046071809999999998 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429077+0000", 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3615999999999994e-05 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429209+0000", 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199699999999999 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43049 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130034655999999, 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492449999999998, 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43049 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382936+0000", 2024-09-21T14:22:19.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382935+0000", 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382937+0000", 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1550000000000001e-06 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382943+0000", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382947+0000", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0670000000000002e-06 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429235+0000", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046287933000000003 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429308+0000", 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2879000000000005e-05 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429428+0000", 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120269 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43050 3.15 3:ab49d51f:::rbd_data.112168cdcfd1.0000000000000003:15 [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.130019386000001, 2024-09-21T14:22:19.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046740351999999999, 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43050 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382951+0000", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.382968+0000", 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7289e-05 2024-09-21T14:22:19.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383094+0000", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012638999999999999 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383114+0000", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0081999999999998e-05 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429454+0000", 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046339578999999999 2024-09-21T14:22:19.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429527+0000", 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3450999999999999e-05 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429691+0000", 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016386100000000001 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.576 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43051 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129789567000003, 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046543382000000001, 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43051 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:19.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383181+0000", 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383180+0000", 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383184+0000", 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7510000000000002e-06 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383207+0000", 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2908000000000002e-05 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383231+0000", 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3981999999999998e-05 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429717+0000", 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046485897999999998 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429724+0000", 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5039999999999996e-06 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43052 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129705469000001, 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046500398999999998, 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43052 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:19.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383265+0000", 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383264+0000", 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383281+0000", 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6515e-05 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383287+0000", 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4439999999999996e-06 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383294+0000", 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1680000000000001e-06 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429747+0000", 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046453853000000003 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429765+0000", 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7918999999999999e-05 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.584 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43053 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129671178999999, 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492406, 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43053 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383299+0000", 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383301+0000", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3580000000000001e-06 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383307+0000", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9619999999999998e-06 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383311+0000", 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:19.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429785+0000", 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046474161 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429792+0000", 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.139e-06 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43054 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129655401999997, 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046502169000000003, 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43054 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383315+0000", 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383317+0000", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9709999999999998e-06 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383322+0000", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5589999999999996e-06 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383328+0000", 2024-09-21T14:22:19.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429810+0000", 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046481947000000003 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429817+0000", 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43055 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129638305999997, 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046513090999999999, 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43055 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383332+0000", 2024-09-21T14:22:19.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383337+0000", 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383342+0000", 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4559999999999999e-06 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383347+0000", 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3830000000000002e-06 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429839+0000", 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046492377000000001 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429845+0000", 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1689999999999997e-06 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43056 3.15 3.f8ab92d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.129619953999999, 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046522758999999997, 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43056 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.597 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383350+0000", 2024-09-21T14:22:19.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383352+0000", 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383357+0000", 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5380000000000002e-06 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.383362+0000", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2629999999999997e-06 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429865+0000", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046503567000000003 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.429873+0000", 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.858e-06 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.600 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.601 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.601 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:19.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:19.603 INFO:teuthology.orchestra.run.smithi073.stdout:flattening image image_client.0-clone39 2024-09-21T14:22:19.604 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:19.602+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:19.703 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:19.703+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:19.754 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-21T14:22:19.757 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.757 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:19.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:19.757 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:19.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:19.804 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:19.804+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:19.873 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.874 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:19.874 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:19.874 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:19.874 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:19.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:19.926 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:19.926+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:19.985 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:19.985 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:19.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.764059025000002, 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.986 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:19.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:19.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:19.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42293 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3274752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.921644026000003, 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032075181000000001, 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42293 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063352+0000", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063351+0000", 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063354+0000", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0880000000000002e-06 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063359+0000", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7620000000000001e-06 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063364+0000", 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3560000000000003e-06 2024-09-21T14:22:19.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095339+0000", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031974925000000001 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095389+0000", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0095000000000001e-05 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095427+0000", 2024-09-21T14:22:19.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8389000000000001e-05 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:42294 3.1c 3:3eec6681:::rbd_data.1121cb9dc356.0000000000000000:13 [read 3500032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.921628421000001, 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032160279, 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:19.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 42294 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063367+0000", 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063369+0000", 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0320000000000002e-06 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:19.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063375+0000", 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4729999999999999e-06 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.063379+0000", 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5329999999999996e-06 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095440+0000", 2024-09-21T14:22:19.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032060937999999997 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095489+0000", 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8711000000000002e-05 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.095528+0000", 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8926e-05 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:19.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53665 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272267968000001, 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032083390000000003, 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53665 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712728+0000", 2024-09-21T14:22:20.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712733+0000", 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712740+0000", 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9290000000000001e-06 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712746+0000", 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744674+0000", 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031927981000000001 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744749+0000", 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744811+0000", 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2681000000000005e-05 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272234563000001, 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:20.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272216859, 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.012 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272201188, 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:20.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:20.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:20.023 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272185673999999, 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:20.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:20.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272170607, 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:20.032 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272155412, 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:20.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:20.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272140306000001, 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.040 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272125196000001, 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:20.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:20.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272109831000002, 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.045 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:20.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:20.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272094372000002, 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272079339000001, 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.053 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272063450000001, 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:20.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:20.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272037774000001, 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:20.061 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:20.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:20.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:20.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272022489000001, 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:20.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.272000524999999, 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.070 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:20.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.271984541999998, 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.078 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:20.080 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:20.080+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:20.180 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:20.180+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:20.223 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:20.224 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.232 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:20.280 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:20.281+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:20.334 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.334 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:20.334 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:20.334 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:20.334 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:20.386 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:20.386+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.446 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.053294158999996, 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.053283258, 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.451 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:20.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:20.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.053272849999999, 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:20.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:20.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.053264908999999, 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:20.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:20.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976526923999998, 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:20.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:20.468 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976508514000002, 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:20.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976482765, 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:20.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976461735999997, 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.477 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:20.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976435885999997, 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.485 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976417407, 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:20.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:20.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.489 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976399284999999, 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:20.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976378922999999, 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.494 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:20.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:20.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976359231000004, 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.498 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976342252999999, 2024-09-21T14:22:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976322580999998, 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976293118000001, 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976250872000001, 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976231884000001, 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976208526000001, 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.976185172999998, 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.532 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:22:20.534 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:20.533+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:20.634 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:20.634+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:20.675 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:20.675 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:20.768 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:20.768+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:20.826 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.826 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:20.826 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:20.826 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:20.826 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:20.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:20.875 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:20.875+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:20.936 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:20.936+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.978 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 71.435779925000006, 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.979 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:20.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:20.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:20.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.987 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43094 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590253760000003, 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355065000000002, 2024-09-21T14:22:20.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.990 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:20.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.990 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43094 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:20.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387465+0000", 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387470+0000", 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5480000000000003e-06 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387475+0000", 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1560000000000003e-06 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387479+0000", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7330000000000001e-06 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427814+0000", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040335226000000002 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427821+0000", 2024-09-21T14:22:20.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6889999999999998e-06 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43095 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590236984999997, 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040366234000000001, 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:20.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43095 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387482+0000", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387484+0000", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.857e-06 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387489+0000", 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:20.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387493+0000", 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2830000000000003e-06 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427842+0000", 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040348342000000002 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:20.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427849+0000", 2024-09-21T14:22:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9759999999999998e-06 2024-09-21T14:22:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:20.999 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590222431999997, 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:22:21.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:22:21.003 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590203563000003, 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.004 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:21.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:21.008 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590189531, 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:21.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:21.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590173532000001, 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.013 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:21.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590158635999998, 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.017 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:21.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590143699999999, 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.021 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.024 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590128720000003, 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.025 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:21.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590114104000001, 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:21.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.032 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590099057000003, 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.033 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:21.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590084498000003, 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590069773000003, 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.041 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:21.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590055333000002, 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:21.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590040983000002, 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.049 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590020916999997, 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.053 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:21.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.590006006999999, 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.589991067, 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.061 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.589971374000001, 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:21.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:21.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:21.069 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:21.070 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:21.071 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:21.071+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:21.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:21.185 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:21.186 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:21.186 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:21.186 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:21.194 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:21.194+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:21.195 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:21.269 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:21.269 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:21.271 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:21.272+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:21.285 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:21.285 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:21.299 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:21.299 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:21.300 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:21.300 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:21.307 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:21.316 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:21.316 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:21.332 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:21.332 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:21.348 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:21.348 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:21.364 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:21.364 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:21.372 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:21.372+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:21.380 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:21.380 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:21.451 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:21.451 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:21.459 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:21.472 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:21.473+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:21.563 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:21.564 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:21.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:21.573 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:21.573+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:21.675 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:21.675+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:21.684 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:21.685 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:21.685 INFO:teuthology.orchestra.run.smithi073.stdout:5525 read 0x1d05c6f thru 0x1d0afe5 (0x5377 bytes) 2024-09-21T14:22:21.685 INFO:teuthology.orchestra.run.smithi073.stdout:5527 write 0x78d80d9 thru 0x78de654 (0x657c bytes) 2024-09-21T14:22:21.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:21.696 INFO:teuthology.orchestra.run.smithi073.stdout:5528 read 0x74114 thru 0x7cd2c (0x8c19 bytes) 2024-09-21T14:22:21.696 INFO:teuthology.orchestra.run.smithi073.stdout:5529 write 0x2e37ff3 thru 0x2e40f76 (0x8f84 bytes) 2024-09-21T14:22:21.705 INFO:teuthology.orchestra.run.smithi073.stdout:5530 read 0x938e7ba thru 0x9399467 (0xacae bytes) 2024-09-21T14:22:21.706 INFO:teuthology.orchestra.run.smithi073.stdout:5531 write 0x33d3a16 thru 0x33d4aa4 (0x108f bytes) 2024-09-21T14:22:21.709 INFO:teuthology.orchestra.run.smithi073.stdout:5532 punch from 0x5c0bf01 to 0x5c192d3, (0xd3d2 bytes) 2024-09-21T14:22:21.709 INFO:teuthology.orchestra.run.smithi073.stdout:5533 read 0x49fe7a8 thru 0x4a0c5b5 (0xde0e bytes) 2024-09-21T14:22:21.709 INFO:teuthology.orchestra.run.smithi073.stdout:5536 read 0x182d94f thru 0x183bdb8 (0xe46a bytes) 2024-09-21T14:22:21.713 INFO:teuthology.orchestra.run.smithi073.stdout:5537 trunc from 0xd93bb5a to 0xa952ab7 2024-09-21T14:22:21.723 INFO:teuthology.orchestra.run.smithi073.stdout:5538 writesame 0x612c552 thru 0x6130207 data_size 0x13(0x3cb6 bytes) 2024-09-21T14:22:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:5539 writesame 0x80539fc thru 0x805dfa7 data_size 0x2e(0xa5ac bytes) 2024-09-21T14:22:21.734 INFO:teuthology.orchestra.run.smithi073.stdout:5540 clone 41 order 25 su 2097152 sc 4 2024-09-21T14:22:21.781 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:21.781+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:21.845 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:21.846 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:21.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:21.877 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:21.877+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:21.977 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:21.978+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:21.997 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:21.997 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.005 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:22.078 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:22.078+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:22.107 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.108 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.116 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:22.178 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:22.178+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:22.260 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.260 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.269 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:22.312 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:22.313+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:22.372 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.372 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:22.381 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:22.380+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:22.481 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:22.482+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:22.523 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.523 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.533 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:22.534 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:22:22.582 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:22.582+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:22.589 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone40 intact 2024-09-21T14:22:22.636 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.637 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:22.689 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:22.689+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:22.714 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-21T14:22:22.758 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.758 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:22.784 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:22.784+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:22.885 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:22.886+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:22.914 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:22.914 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:22.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:23.007 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:23.007+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:23.066 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:23.067 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:23.075 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:23.116 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:23.116+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:23.187 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:23.188+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:23.217 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:23.217 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:23.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:23.310 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:23.310+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:23.368 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:23.368 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:23.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:23.389 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:23.390+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:23.392 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:23.490 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:23.490+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:23.523 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:23.523 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:23.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:22:23.539 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:23.539 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:23.557 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:23.557 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:23.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:23.575 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:23.591 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:23.592+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:23.594 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:23.595 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:23.611 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:23.627 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:23.644 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:23.644 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:23.647 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:23.648 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:23.648 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:23.648 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:23.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:23.661 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:23.661 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:23.692 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:23.693+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:23.734 INFO:teuthology.orchestra.run.smithi073.stdout:5541 writesame 0x4014927 thru 0x4022e7d data_size 0x1(0xe557 bytes) 2024-09-21T14:22:23.784 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:23.784 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:23.784 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:23.784 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:23.784 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:23.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:23.794 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:23.793+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:23.848 INFO:teuthology.orchestra.run.smithi073.stdout:5542 punch from 0x7a918f to 0x7ac843, (0x36b4 bytes) 2024-09-21T14:22:23.854 INFO:teuthology.orchestra.run.smithi073.stdout:5543 write 0xadd094a thru 0xadd6d4c (0x6403 bytes) 2024-09-21T14:22:23.861 INFO:teuthology.orchestra.run.smithi073.stdout:5545 read 0x8401506 thru 0x840474c (0x3247 bytes) 2024-09-21T14:22:23.861 INFO:teuthology.orchestra.run.smithi073.stdout:5546 read 0x375b586 thru 0x376119c (0x5c17 bytes) 2024-09-21T14:22:23.861 INFO:teuthology.orchestra.run.smithi073.stdout:5554 write 0x57e9641 thru 0x57ed965 (0x4325 bytes) 2024-09-21T14:22:23.894 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:23.894+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:23.943 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.550958010000002, 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.550947108999999, 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.550936700999998, 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.550928759999998, 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474190774999997, 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474172365000001, 2024-09-21T14:22:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474146615999999, 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474125587000003, 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:23.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:23.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.979 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474099737000003, 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:23.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:23.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474081257999998, 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:23.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:23.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474063135999998, 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:23.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474042773999997, 2024-09-21T14:22:23.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474023082000002, 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:23.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:23.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:23.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:24.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.474006103999997, 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:24.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.005 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473986431999997, 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:24.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.009 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473956969, 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.010 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:24.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:24.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473914723, 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.014 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473895734999999, 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.018 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:24.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473872376999999, 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:24.022 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:24.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:24.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:24.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.473849024000003, 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.026 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:24.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.030 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:24.031 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:24.031 INFO:teuthology.orchestra.run.smithi073.stdout:5556 punch from 0x2f05661 to 0x2f0ba6c, (0x640b bytes) 2024-09-21T14:22:24.031 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:24.031 INFO:teuthology.orchestra.run.smithi073.stdout:5558 trunc from 0xadd6d4d to 0xc1bebe7 2024-09-21T14:22:24.032 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5559 write 0xd304e38 thru 0xd30691a (0x1ae3 bytes) 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5560 punch from 0x1b6192b to 0x1b713c4, (0xfa99 bytes) 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5561 punch from 0x1864056 to 0x18711c4, (0xd16e bytes) 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5562 write 0x40f083b thru 0x40fd0d3 (0xc899 bytes) 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5564 punch from 0x29c148a to 0x29cfb35, (0xe6ab bytes) 2024-09-21T14:22:24.033 INFO:teuthology.orchestra.run.smithi073.stdout:5566 write 0x51ff5e thru 0x524505 (0x45a8 bytes) 2024-09-21T14:22:24.033 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:24.031+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:24.088 INFO:teuthology.orchestra.run.smithi073.stdout:5567 punch from 0x21dc181 to 0x21e0daf, (0x4c2e bytes) 2024-09-21T14:22:24.088 INFO:teuthology.orchestra.run.smithi073.stdout:5568 read 0x92dfb66 thru 0x92e7066 (0x7501 bytes) 2024-09-21T14:22:24.089 INFO:teuthology.orchestra.run.smithi073.stdout:5569 write 0x8833bd1 thru 0x883bb15 (0x7f45 bytes) 2024-09-21T14:22:24.120 INFO:teuthology.orchestra.run.smithi073.stdout:5573 writesame 0x41beafc thru 0x41c11f1 data_size 0x1(0x26f6 bytes) 2024-09-21T14:22:24.125 INFO:teuthology.orchestra.run.smithi073.stdout:5574 read 0x3b51c15 thru 0x3b5a72e (0x8b1a bytes) 2024-09-21T14:22:24.125 INFO:teuthology.orchestra.run.smithi073.stdout:5575 write 0x55b342c thru 0x55b9340 (0x5f15 bytes) 2024-09-21T14:22:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:5576 read 0xb6ddba0 thru 0xb6e2fec (0x544d bytes) 2024-09-21T14:22:24.127 INFO:teuthology.orchestra.run.smithi073.stdout:5577 read 0x5c1f906 thru 0x5c2b3a7 (0xbaa2 bytes) 2024-09-21T14:22:24.128 INFO:teuthology.orchestra.run.smithi073.stdout:5579 write 0x1332028 thru 0x1335dce (0x3da7 bytes) 2024-09-21T14:22:24.132 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:24.132+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.043534536999999998, 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043557435999999998, 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:22:24.193 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:24.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:24.202 INFO:teuthology.orchestra.run.smithi073.stdout:5581 write 0x41de61c thru 0x41e253d (0x3f22 bytes) 2024-09-21T14:22:24.204 INFO:teuthology.orchestra.run.smithi073.stdout:5582 writesame 0x6e7b83 thru 0x6ec496 data_size 0x6(0x4914 bytes) 2024-09-21T14:22:24.206 INFO:teuthology.orchestra.run.smithi073.stdout:5583 punch from 0x732f4b9 to 0x732f664, (0x1ab bytes) 2024-09-21T14:22:24.207 INFO:teuthology.orchestra.run.smithi073.stdout:5584 read 0x75aac05 thru 0x75b6262 (0xb65e bytes) 2024-09-21T14:22:24.207 INFO:teuthology.orchestra.run.smithi073.stdout:5585 read 0xc73e95b thru 0xc7459bc (0x7062 bytes) 2024-09-21T14:22:24.207 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:24.207 INFO:teuthology.orchestra.run.smithi073.stdout:5589 write 0x297b033 thru 0x297c6dd (0x16ab bytes) 2024-09-21T14:22:24.209 INFO:teuthology.orchestra.run.smithi073.stdout:5591 writesame 0xab8171c thru 0xab85db2 data_size 0x1(0x4697 bytes) 2024-09-21T14:22:24.217 INFO:teuthology.orchestra.run.smithi073.stdout:5592 read 0x9caf022 thru 0x9cb38ca (0x48a9 bytes) 2024-09-21T14:22:24.225 INFO:teuthology.orchestra.run.smithi073.stdout:5594 trunc from 0xd30691b to 0x6cda75f 2024-09-21T14:22:24.232 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:24.232+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:24.256 INFO:teuthology.orchestra.run.smithi073.stdout:5596 write 0xc676ed8 thru 0xc67eca7 (0x7dd0 bytes) 2024-09-21T14:22:24.259 INFO:teuthology.orchestra.run.smithi073.stdout:5598 write 0x812c685 thru 0x8131c1f (0x559b bytes) 2024-09-21T14:22:24.265 INFO:teuthology.orchestra.run.smithi073.stdout:5599 read 0x8962fb1 thru 0x896b9c4 (0x8a14 bytes) 2024-09-21T14:22:24.265 INFO:teuthology.orchestra.run.smithi073.stdout:5600 read 0x49ee5ce thru 0x49fb98a (0xd3bd bytes) 2024-09-21T14:22:24.266 INFO:teuthology.orchestra.run.smithi073.stdout:5601 writesame 0x1c801d1 thru 0x1c87695 data_size 0x1(0x74c5 bytes) 2024-09-21T14:22:24.279 INFO:teuthology.orchestra.run.smithi073.stdout:5602 read 0x6445682 thru 0x644ef35 (0x98b4 bytes) 2024-09-21T14:22:24.280 INFO:teuthology.orchestra.run.smithi073.stdout:5603 read 0x67b28bc thru 0x67b3774 (0xeb9 bytes) 2024-09-21T14:22:24.280 INFO:teuthology.orchestra.run.smithi073.stdout:5604 clone 42 order 25 su 8388608 sc 5 2024-09-21T14:22:24.308 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:24.308 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:24.308 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:24.309 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:24.309 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:24.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:24.333 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:24.333+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:24.433 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:24.433+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.918810687999994, 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43096 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073253194999999, 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040377241000000001, 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43096 2024-09-21T14:22:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387497+0000", 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387502+0000", 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8200000000000002e-06 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387508+0000", 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6729999999999997e-06 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387512+0000", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0520000000000003e-06 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427868+0000", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040355917999999998 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427874+0000", 2024-09-21T14:22:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0990000000000004e-06 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073234325999998, 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073220294000002, 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073204294999996, 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:24.483 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073189399, 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:24.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.487 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073174463000001, 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.491 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073159482999998, 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:24.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073144867000003, 2024-09-21T14:22:24.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:24.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073129819999998, 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:24.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073115260999998, 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:24.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:24.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073100535999998, 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073086095999997, 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073071745999997, 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073051679999999, 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:24.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:24.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073036770000002, 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:24.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:24.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073021830000002, 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:24.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.073002137000003, 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.42552124800000002, 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:24.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:22:24.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.31537206499999998, 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:24.538 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:22:24.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:24.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:24.544 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:24.545 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:24.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:24.545+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:24.604 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:24.604 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:24.605 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 3, 4, 5, 6, 7] 2024-09-21T14:22:24.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-21T14:22:24.607 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone41 intact 2024-09-21T14:22:24.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:24.646+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:24.652 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:24.652 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:24.652 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:24.652 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:24.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:24.746 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:24.746+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:24.761 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-21T14:22:24.805 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:24.806 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:24.806 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:24.806 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:24.806 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:24.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:24.847 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:24.847+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:24.922 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:22:24.922+0000 7f0ce787f640 -1 mon.a@0(leader).osd e78 definitely_dead 0 2024-09-21T14:22:24.947 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:24.947+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1953219050000001, 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.963 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:22:24.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:22:24.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1952453949999997, 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:22:24.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.972 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1944354309999996, 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.973 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:22:24.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.194408374, 2024-09-21T14:22:24.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:22:24.981 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1939601509999997, 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:22:24.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:22:24.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.986 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1939378109999996, 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:22:24.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:22:24.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1938961199999998, 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.991 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:22:24.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:22:24.995 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1937374549999999, 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:22:24.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:22:24.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:24.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.000 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1937050339999997, 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:22:25.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:22:25.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:25.005 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.193676505, 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:22:25.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.009 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1936554089999998, 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.010 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1936232389999999, 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.014 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1935467500000003, 2024-09-21T14:22:25.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:22:25.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1935238100000003, 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:22:25.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:22:25.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1935067930000001, 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.027 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:22:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.031 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1933579400000003, 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:22:25.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:22:25.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:22:25.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1933422299999998, 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:25.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:22:25.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:22:25.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:22:25.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.193326807, 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:22:25.041 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:22:25.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:22:25.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1931947420000002, 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.046 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:25.050 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.1931596850000004, 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.055 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:5606 write 0xe1b3538 thru 0xe1b3edd (0x9a6 bytes) 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:5608 read 0x3c7bcce thru 0x3c7cfe2 (0x1315 bytes) 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:5609 read 0xddbe12 thru 0xde7b79 (0xbd68 bytes) 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:5610 read 0xe044490 thru 0xe04ca56 (0x85c7 bytes) 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:25.057 INFO:teuthology.orchestra.run.smithi073.stdout:5612 writesame 0x8b43a00 thru 0x8b45ccf data_size 0x4(0x22d0 bytes) 2024-09-21T14:22:25.058 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:25.058 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:25.056+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:25.095 INFO:teuthology.orchestra.run.smithi073.stdout:5613 write 0xccd5766 thru 0xcce11fd (0xba98 bytes) 2024-09-21T14:22:25.104 INFO:teuthology.orchestra.run.smithi073.stdout:5614 writesame 0xe375310 thru 0xe37d960 data_size 0x41(0x8651 bytes) 2024-09-21T14:22:25.105 INFO:teuthology.orchestra.run.smithi073.stdout:5616 read 0x5f268af thru 0x5f2c325 (0x5a77 bytes) 2024-09-21T14:22:25.106 INFO:teuthology.orchestra.run.smithi073.stdout:5618 write 0x94c8f6f thru 0x94cdbc3 (0x4c55 bytes) 2024-09-21T14:22:25.157 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:25.157+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:25.174 INFO:teuthology.orchestra.run.smithi073.stdout:5619 read 0x9397069 thru 0x939e9ee (0x7986 bytes) 2024-09-21T14:22:25.175 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:25.175 INFO:teuthology.orchestra.run.smithi073.stdout:5622 write 0xca620be thru 0xca67ae7 (0x5a2a bytes) 2024-09-21T14:22:25.177 INFO:teuthology.orchestra.run.smithi073.stdout:5623 writesame 0xd3d61fb thru 0xd3e5568 data_size 0x1(0xf36e bytes) 2024-09-21T14:22:25.206 INFO:teuthology.orchestra.run.smithi073.stdout:5624 read 0x30bc592 thru 0x30bf326 (0x2d95 bytes) 2024-09-21T14:22:25.207 INFO:teuthology.orchestra.run.smithi073.stdout:5625 writesame 0xdbf5e45 thru 0xdc02e4c data_size 0x54(0xd008 bytes) 2024-09-21T14:22:25.211 INFO:teuthology.orchestra.run.smithi073.stdout:5626 writesame 0x569259 thru 0x5789c5 data_size 0x1(0xf76d bytes) 2024-09-21T14:22:25.218 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:25.219 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:25.219 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:25.219 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:25.258 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:25.258+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:25.290 INFO:teuthology.orchestra.run.smithi073.stdout:5627 read 0xb60ed5e thru 0xb61b954 (0xcbf7 bytes) 2024-09-21T14:22:25.291 INFO:teuthology.orchestra.run.smithi073.stdout:5629 read 0xac14888 thru 0xac1c08a (0x7803 bytes) 2024-09-21T14:22:25.291 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:25.297 INFO:teuthology.orchestra.run.smithi073.stdout:5631 write 0xe76f013 thru 0xe77efe8 (0xffd6 bytes) 2024-09-21T14:22:25.298 INFO:teuthology.orchestra.run.smithi073.stdout:5632 clone 43 order 23 su 262144 sc 4 2024-09-21T14:22:25.358 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:25.359+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:25.369 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:25.370 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:25.370 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:25.370 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:25.370 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:25.463 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:25.463+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:25.522 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.812009832000001, 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.523 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:25.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:22:25.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:22:25.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811993452000003, 2024-09-21T14:22:25.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:22:25.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:22:25.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.531 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811984224, 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.532 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:22:25.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811972828000002, 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.536 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:22:25.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811950101000001, 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811933844999999, 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:22:25.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:22:25.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811919090000004, 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.549 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:22:25.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.553 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811910036, 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:25.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.557 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811898992000003, 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.558 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811883590999997, 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.566 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811866893000001, 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:25.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:22:25.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811851286, 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811834163999997, 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:22:25.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:25.579 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811815553000002, 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:22:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:22:25.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811796223000002, 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:22:25.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.811776586000001, 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:22:25.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:22:25.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80476 3.1a e77/19 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head v 77'391, mlcod=72'389)", 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:25.592 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.718745414, 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.046247176000000001, 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833300+0000", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03008779 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833325+0000", 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5609000000000002e-05 2024-09-21T14:22:25.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833334+0000", 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8470000000000002e-06 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833415+0000", 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1089999999999998e-05 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833442+0000", 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7087999999999999e-05 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.849411+0000", 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015968755000000001 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.849459+0000", 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8173000000000001e-05 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80693 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22976065~17189] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.6431552620000001, 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.067724593999999999, 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80693 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.597 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:25.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909741+0000", 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.030939548000000001 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909769+0000", 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7597e-05 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909778+0000", 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0110000000000006e-06 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909870+0000", 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2313999999999997e-05 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909990+0000", 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000120142 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 7", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.911158+0000", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0011675450000000001 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.941402+0000", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.030244018000000001 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946425+0000", 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0050235100000000001 2024-09-21T14:22:25.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946480+0000", 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5102000000000002e-05 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946527+0000", 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6588000000000002e-05 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 1.5203475900000001, 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:25.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:22:25.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:22:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.608 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:82787 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30183279~19541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.39950170800000001, 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.053140573000000003, 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 82787 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122454+0000", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143192+0000", 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020738090000000001 2024-09-21T14:22:25.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143217+0000", 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5465000000000001e-05 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143243+0000", 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6103999999999999e-05 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:25.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143313+0000", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9871999999999998e-05 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143496+0000", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000182888 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 7", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145543+0000", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0020468069999999999 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:25.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175066+0000", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029523309000000001 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175490+0000", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000423941 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175551+0000", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0519000000000001e-05 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175596+0000", 2024-09-21T14:22:25.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5920000000000001e-05 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:25.614 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:22:25.616 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.3. 2024-09-21T14:22:25.616 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:25.617 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:25.616+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:25.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:25.743 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:25.743+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:25.758 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:25.758 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:25.759 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:25.759 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:25.817 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:25.818+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:25.836 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:25.837 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:25.857 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:25.868 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:25.868 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:25.868 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:25.868 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:25.868 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:25.873 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:25.873 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:25.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:25.889 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:25.889 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:25.905 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:25.905 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:25.921 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:25.921 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:25.921 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:25.922+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:25.937 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:25.953 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:25.953 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:25.981 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:25.981 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:25.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:26.031 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:26.031+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:26.120 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:26.120+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:26.133 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.133 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.136 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:22:26.144 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:26.220 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:26.220+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:26.246 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.247 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.255 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:26.340 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:26.340+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.399 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:26.451 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:26.451+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:26.510 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.510 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:26.560 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:26.560+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:26.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:26.624+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:26.661 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.661 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.670 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:26.725 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:26.726+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:26.772 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.772 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:26.826 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:26.826+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:26.883 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:26.884 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:26.892 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:26.933 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:26.933+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:27.028 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:27.028+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:27.034 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.034 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:27.129 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:27.129+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:27.153 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.153 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:27.230 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:27.230+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:27.283 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.283 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:27.331 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:27.331+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:27.416 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.416 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:27.471 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:27.471+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:27.533 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:27.533+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:27.543 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.543 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.553 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:27.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:27.634+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:27.713 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.713 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:27.735 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:27.735+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:27.835 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:27.835+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:27.842 INFO:teuthology.orchestra.run.smithi073.stdout:leaving image image_client.0-clone42 intact 2024-09-21T14:22:27.845 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.845 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.857 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:27.936 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:27.936+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:27.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:27.971 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:27.971 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:27.983 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:28.037 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:28.037+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:28.042 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-21T14:22:28.095 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:28.096 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:28.105 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:28.105 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:28.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:28.123 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:28.123 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:28.137 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:28.137+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:28.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:28.140 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:28.152 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:22:28.164 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:28.183 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:28.183 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:28.200 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:28.201 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:28.217 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:28.217 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:28.228 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:28.228 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:28.228 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:28.228 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:28.228 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:28.233 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:28.233 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:28.237 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:28.238 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:28.238+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:28.338 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:28.339+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:28.340 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5727182170000003, 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:22:28.342 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.343 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.344 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5726417070000007, 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.345 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:28.346 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.347 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:22:28.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5718317430000006, 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.350 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.351 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.352 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:22:28.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5718046860000001, 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.355 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.357 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.358 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5713564630000008, 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.359 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.360 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:22:28.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:22:28.363 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5713341229999997, 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.364 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.366 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:22:28.367 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.368 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5712924319999999, 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.369 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:22:28.370 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.371 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.372 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.373 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5711337669999992, 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:22:28.375 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.376 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5711013460000007, 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.378 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.381 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:28.382 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5710728169999992, 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:22:28.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5710517209999999, 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.387 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:28.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.391 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5710195509999991, 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.392 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:22:28.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.394 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5709430619999996, 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:22:28.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5709201220000004, 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.401 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:28.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5709031049999993, 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5707542520000004, 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.410 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:22:28.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:22:28.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5707385420000008, 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5707231190000002, 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.419 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:28.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5705910539999994, 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:22:28.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.5705559969999996, 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:28.428 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:22:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:28.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.432 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:28.434 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:28.439 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:28.439+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:28.540 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:28.540+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:28.578 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:28.578 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:28.578 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:28.578 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:28.586 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:28.671 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:28.671+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:28.730 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:28.730 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:28.730 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:28.730 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:28.730 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:28.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:28.742 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:28.743+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:28.843 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:28.843+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:28.882 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10551 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171663514999999, 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033537971, 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.883 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10551 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709948+0000", 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709953+0000", 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709956+0000", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2399999999999999e-06 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709962+0000", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743292+0000", 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033330588000000001 2024-09-21T14:22:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743386+0000", 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3572000000000005e-05 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743486+0000", 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.9821999999999994e-05 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10553 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171647135000001, 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033700607, 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10553 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.888 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709964+0000", 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709965+0000", 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1480000000000001e-06 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709968+0000", 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8899999999999999e-06 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709970+0000", 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2749999999999998e-06 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743505+0000", 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033535080000000002 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743580+0000", 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5130999999999996e-05 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743665+0000", 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4291999999999999e-05 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10558 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171637906999997, 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033911279000000003, 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10558 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709973+0000", 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709975+0000", 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.719e-06 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709978+0000", 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6459999999999999e-06 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709981+0000", 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4899999999999999e-06 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743699+0000", 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033718388000000002 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743784+0000", 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4753000000000007e-05 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743885+0000", 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00010055700000000001 2024-09-21T14:22:28.897 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10560 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:28.898 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171626510999999, 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034098718, 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.899 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10560 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709985+0000", 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709984+0000", 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709986+0000", 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0150000000000002e-06 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.709992+0000", 2024-09-21T14:22:28.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710002+0000", 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.8570000000000007e-06 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743920+0000", 2024-09-21T14:22:28.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.033918116999999998 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.743997+0000", 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7088000000000005e-05 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744084+0000", 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6446000000000005e-05 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.905 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10563 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171603783999998, 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034275018999999997, 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.906 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10563 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:28.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710008+0000", 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710007+0000", 2024-09-21T14:22:28.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710009+0000", 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0829999999999998e-06 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710015+0000", 2024-09-21T14:22:28.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710020+0000", 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744104+0000", 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034084272999999998 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.911 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.912 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744188+0000", 2024-09-21T14:22:28.912 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3832999999999995e-05 2024-09-21T14:22:28.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.912 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.913 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744283+0000", 2024-09-21T14:22:28.913 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4890999999999997e-05 2024-09-21T14:22:28.913 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10565 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171587528000003, 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034449157000000001, 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.914 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10565 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710024+0000", 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710023+0000", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710030+0000", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5130000000000002e-06 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710034+0000", 2024-09-21T14:22:28.916 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7419999999999999e-06 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710036+0000", 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5469999999999999e-06 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744315+0000", 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034279085000000001 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.917 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744391+0000", 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5961e-05 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744473+0000", 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1737999999999996e-05 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10571 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171572773000001, 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034640531000000002, 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10571 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.919 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710038+0000", 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710039+0000", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1680000000000001e-06 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710042+0000", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.005e-06 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710045+0000", 2024-09-21T14:22:28.921 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3010000000000001e-06 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744507+0000", 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034461823000000003 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744588+0000", 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1009999999999999e-05 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.922 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744679+0000", 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1436999999999999e-05 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10573 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171563718999998, 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034838239, 2024-09-21T14:22:28.923 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10573 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.924 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710048+0000", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710047+0000", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710049+0000", 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.235e-06 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.925 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710054+0000", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0159999999999999e-06 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710056+0000", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744713+0000", 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034657540000000001 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744793+0000", 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9788000000000002e-05 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744886+0000", 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2579999999999998e-05 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10577 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171552675000001, 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035046076000000002, 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10577 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.928 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710059+0000", 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710058+0000", 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.929 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710060+0000", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.643e-06 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710065+0000", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4199999999999998e-06 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710070+0000", 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1869999999999999e-06 2024-09-21T14:22:28.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.744926+0000", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.034856442000000001 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745013+0000", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6485999999999998e-05 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745105+0000", 2024-09-21T14:22:28.931 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2200999999999994e-05 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10579 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171537274000002, 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035135917000000003, 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.932 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10579 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710074+0000", 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710076+0000", 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.846e-06 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.934 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710082+0000", 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1800000000000001e-06 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710087+0000", 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9690000000000002e-06 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745116+0000", 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035029751999999997 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745162+0000", 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5454999999999998e-05 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745210+0000", 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8059999999999997e-05 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10583 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171520575999999, 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035324704999999998, 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10583 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.937 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710091+0000", 2024-09-21T14:22:28.938 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710090+0000", 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710092+0000", 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.838e-06 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710098+0000", 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.940 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710103+0000", 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.408e-06 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745242+0000", 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035139468 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745325+0000", 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3265999999999999e-05 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.941 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745415+0000", 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0074999999999994e-05 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10584 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171504968999997, 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035435084999999998, 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.942 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10584 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710106+0000", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710114+0000", 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6129999999999998e-06 2024-09-21T14:22:28.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710118+0000", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0269999999999997e-06 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710121+0000", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2550000000000002e-06 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745442+0000", 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035321183999999999 2024-09-21T14:22:28.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745487+0000", 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5179999999999998e-05 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745541+0000", 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4104999999999998e-05 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10588 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171487847000002, 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035619352, 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10588 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710123+0000", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710124+0000", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.142e-06 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710128+0000", 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7170000000000002e-06 2024-09-21T14:22:28.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710139+0000", 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1106e-05 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745575+0000", 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035436282 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.950 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745656+0000", 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0401999999999994e-05 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745743+0000", 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10590 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:28.951 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171469236, 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035733000000000001, 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10590 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:28.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710142+0000", 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710144+0000", 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8330000000000001e-06 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710153+0000", 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.217e-06 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710157+0000", 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5109999999999997e-06 2024-09-21T14:22:28.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745770+0000", 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035612272 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745815+0000", 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5059999999999999e-05 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745875+0000", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0421000000000003e-05 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10595 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171449905999999, 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035903164000000001, 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.956 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10595 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710161+0000", 2024-09-21T14:22:28.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710163+0000", 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710169+0000", 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7590000000000003e-06 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.959 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710178+0000", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0990000000000005e-06 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745906+0000", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035727795 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.745990+0000", 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4907999999999994e-05 2024-09-21T14:22:28.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746065+0000", 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4093999999999999e-05 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:10597 3.18 3:1c155e9e:::rbd_data.1121cf46bdbd.0000000000000014:7 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.171430268999998, 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.036017755999999998, 2024-09-21T14:22:28.961 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 10597 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710181+0000", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710182+0000", 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3480000000000001e-06 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710187+0000", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.710192+0000", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5800000000000002e-06 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746100+0000", 2024-09-21T14:22:28.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.035908337999999998 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746160+0000", 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9555000000000002e-05 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.746199+0000", 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9060999999999998e-05 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80476 3.1a e77/19 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head v 77'391, mlcod=72'389)", 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.0783990970000001, 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.046247176000000001, 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.803212+0000", 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833300+0000", 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03008779 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833325+0000", 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5609000000000002e-05 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833334+0000", 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8470000000000002e-06 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833415+0000", 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1089999999999998e-05 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.833442+0000", 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7087999999999999e-05 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.849411+0000", 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015968755000000001 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.849459+0000", 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8173000000000001e-05 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.970 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80693 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22976065~17189] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 5.002808945, 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.067724593999999999, 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80693 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.878802+0000", 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909741+0000", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.030939548000000001 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909769+0000", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.7597e-05 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909778+0000", 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0110000000000006e-06 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909870+0000", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2313999999999997e-05 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.909990+0000", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000120142 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 7", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.911158+0000", 2024-09-21T14:22:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0011675450000000001 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.941402+0000", 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.030244018000000001 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946425+0000", 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0050235100000000001 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946480+0000", 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5102000000000002e-05 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946527+0000", 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6588000000000002e-05 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.976 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.8800012729999995, 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:28.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:22:28.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:82787 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30183279~19541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 3.7591553910000002, 2024-09-21T14:22:28.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.053140573000000003, 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:28.983 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 82787 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122456+0000", 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.122454+0000", 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:28.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143192+0000", 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020738090000000001 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143217+0000", 2024-09-21T14:22:28.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5465000000000001e-05 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143243+0000", 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6103999999999999e-05 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143313+0000", 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9871999999999998e-05 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.143496+0000", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000182888 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 7", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145543+0000", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0020468069999999999 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175066+0000", 2024-09-21T14:22:28.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029523309000000001 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175490+0000", 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000423941 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175551+0000", 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0519000000000001e-05 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:28.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.175596+0000", 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5920000000000001e-05 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:28.990 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:28.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:28.992 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:28.992+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:29.093 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:29.093+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:29.135 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.135 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:29.135 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:29.135 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:29.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:29.157 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:22:29.218 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:29.218+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:29.286 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.286 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:29.286 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:29.286 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:29.286 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:29.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:29.295 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:29.295+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:29.348 INFO:teuthology.orchestra.run.smithi073.stdout:5633 read 0x933ac72 thru 0x933e612 (0x39a1 bytes) 2024-09-21T14:22:29.349 INFO:teuthology.orchestra.run.smithi073.stdout:5634 write 0x7c63fc1 thru 0x7c673dc (0x341c bytes) 2024-09-21T14:22:29.365 INFO:teuthology.orchestra.run.smithi073.stdout:5635 clone 44 order 21 su 262144 sc 4 2024-09-21T14:22:29.395 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:29.396+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969729469000001, 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:29.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.406 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969712827000002, 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.407 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969687776999997, 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:29.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:29.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969669625000002, 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.416 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.418 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:29.420 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969646388000001, 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:29.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969625321000002, 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:29.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969608960999999, 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.433 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969578681000002, 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:29.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:29.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969563543, 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:29.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:29.442 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969543946999998, 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:29.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.96953517, 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:29.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:29.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969522808999997, 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.451 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.455 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969513392000003, 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:29.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969492201000001, 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.460 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:29.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.464 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969475093, 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:29.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:29.468 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969458150000001, 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969441771, 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:29.473 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:29.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.477 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969400411000002, 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.478 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:29.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.969380729999997, 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:29.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 36.564492283, 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:29.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.492 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:29.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:29.497 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:29.497+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:29.597 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:29.597+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:29.636 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.637 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:29.637 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:29.637 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:29.645 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:29.729 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:29.729+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:29.787 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.787 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:29.787 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:29.787 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:29.787 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:29.795 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:29.837 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:29.837+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:29.899 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:29.900+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:29.940 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343703015999999, 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.941 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:29.943 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.944 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:29.945 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343685502, 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:29.946 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:29.947 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.948 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343668248, 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.951 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:29.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:29.953 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:29.954 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343640397999998, 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:29.956 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:29.957 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343618794000001, 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343601796000002, 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.963 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:29.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343584776, 2024-09-21T14:22:29.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.968 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:29.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343567325999999, 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.972 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.975 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343550188000002, 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.976 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:29.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343526961000002, 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.980 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:29.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.984 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343510686999998, 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:29.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343486315, 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.989 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343463747999998, 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:29.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:29.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:29.997 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343446827000001, 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:29.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343430327999997, 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:30.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:30.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:30.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343409330999997, 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:30.006 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:30.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 60.343387272999998, 2024-09-21T14:22:30.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:30.014 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.169316401000003, 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:30.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:30.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 6.027083277, 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:30.020 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:22:30.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.024 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 4.794373201, 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:30.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:22:30.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.029 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.029 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.029 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:30.029 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:30.029 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:30.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:30.031 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:30.030+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:30.131 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:30.131+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:30.139 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:30.139 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:30.139 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:30.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:30.232 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:30.232+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:30.238 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone43 from 0xe77efe9 (overlap 0xe77efe9) to 0xb932654 2024-09-21T14:22:30.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:30.306 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:30.307 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:30.307 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:30.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:30.360 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:30.360+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:30.379 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:30.380 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:30.414 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:30.415 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:30.427 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:30.427 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:30.434 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:30.434+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:30.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:30.438 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:30.457 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:30.476 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:30.496 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:30.496 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:30.497 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-21T14:22:30.526 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:30.526 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:30.535 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:30.535+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:30.557 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:30.558 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:30.577 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:30.577 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:30.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:30.617 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:30.618 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:30.618 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 4, 5, 6, 7] 2024-09-21T14:22:30.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-21T14:22:30.636 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:30.636+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:30.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:30.737+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:30.766 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:30.767 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:30.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:30.838 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:30.838+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:30.879 INFO:teuthology.orchestra.run.smithi073.stdout:5640 trunc from 0xe77efe9 to 0xe28c236 2024-09-21T14:22:30.883 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:30.883 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:30.893 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:30.900 INFO:teuthology.orchestra.run.smithi073.stdout:5642 write 0xdb2ba1e thru 0xdb2ed77 (0x335a bytes) 2024-09-21T14:22:30.904 INFO:teuthology.orchestra.run.smithi073.stdout:5643 read 0x1dcec9c thru 0x1dd7676 (0x89db bytes) 2024-09-21T14:22:30.905 INFO:teuthology.orchestra.run.smithi073.stdout:5644 read 0xdaefe9c thru 0xdaf1ef4 (0x2059 bytes) 2024-09-21T14:22:30.905 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:30.905 INFO:teuthology.orchestra.run.smithi073.stdout:5646 writesame 0x16df87b thru 0x16e677e data_size 0x46(0x6f04 bytes) 2024-09-21T14:22:30.918 INFO:teuthology.orchestra.run.smithi073.stdout:5647 writesame 0x9289ae6 thru 0x92993f6 data_size 0x1(0xf911 bytes) 2024-09-21T14:22:30.935 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:22:30.934+0000 7f0ce787f640 -1 mon.a@0(leader).osd e84 definitely_dead 0 2024-09-21T14:22:30.939 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:30.939+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:30.951 INFO:teuthology.orchestra.run.smithi073.stdout:5649 write 0xd2ea1b1 thru 0xd2f5a52 (0xb8a2 bytes) 2024-09-21T14:22:30.956 INFO:teuthology.orchestra.run.smithi073.stdout:5650 punch from 0x3f82ae1 to 0x3f8bf1c, (0x943b bytes) 2024-09-21T14:22:30.956 INFO:teuthology.orchestra.run.smithi073.stdout:5651 read 0x8b27e70 thru 0x8b2a4fa (0x268b bytes) 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:5652 punch from 0x5178b5 to 0x520a5e, (0x91a9 bytes) 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:5655 read 0x914a044 thru 0x914c599 (0x2556 bytes) 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:30.957 INFO:teuthology.orchestra.run.smithi073.stdout:5660 writesame 0x1c8d43 thru 0x1d3484 data_size 0x4f(0xa742 bytes) 2024-09-21T14:22:30.969 INFO:teuthology.orchestra.run.smithi073.stdout:5662 punch from 0xa8b1dfb to 0xa8c1038, (0xf23d bytes) 2024-09-21T14:22:30.975 INFO:teuthology.orchestra.run.smithi073.stdout:5663 write 0xe6aa72e thru 0xe6b8c28 (0xe4fb bytes) 2024-09-21T14:22:30.977 INFO:teuthology.orchestra.run.smithi073.stdout:5666 read 0xa65eca4 thru 0xa667bdb (0x8f38 bytes) 2024-09-21T14:22:30.994 INFO:teuthology.orchestra.run.smithi073.stdout:5668 trunc from 0xe6b8c29 to 0xdf081b 2024-09-21T14:22:31.016 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.016 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:31.040 INFO:teuthology.orchestra.run.smithi073.stdout:5670 writesame 0xc693729 thru 0xc6973e6 data_size 0xa(0x3cbe bytes) 2024-09-21T14:22:31.040 INFO:teuthology.orchestra.run.smithi073.stdout:5671 read 0xb93aaac thru 0xb945f35 (0xb48a bytes) 2024-09-21T14:22:31.040 INFO:teuthology.orchestra.run.smithi073.stdout:5672 write 0x9a31f28 thru 0x9a35a0b (0x3ae4 bytes) 2024-09-21T14:22:31.040 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:31.040+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:31.042 INFO:teuthology.orchestra.run.smithi073.stdout:5673 read 0x9e99b07 thru 0x9ea8ca5 (0xf19f bytes) 2024-09-21T14:22:31.051 INFO:teuthology.orchestra.run.smithi073.stdout:5675 trunc from 0xc6973e7 to 0x4cd26d5 2024-09-21T14:22:31.060 INFO:teuthology.orchestra.run.smithi073.stdout:5676 punch from 0x3fe5081 to 0x3fe51e0, (0x15f bytes) 2024-09-21T14:22:31.060 INFO:teuthology.orchestra.run.smithi073.stdout:5677 punch from 0x1b1581d to 0x1b24402, (0xebe5 bytes) 2024-09-21T14:22:31.060 INFO:teuthology.orchestra.run.smithi073.stdout:5678 read 0x90d555 thru 0x90f7ac (0x2258 bytes) 2024-09-21T14:22:31.060 INFO:teuthology.orchestra.run.smithi073.stdout:5679 punch from 0x2067cfe to 0x206a615, (0x2917 bytes) 2024-09-21T14:22:31.060 INFO:teuthology.orchestra.run.smithi073.stdout:5680 read 0x4b49bec thru 0x4b4cbac (0x2fc1 bytes) 2024-09-21T14:22:31.067 INFO:teuthology.orchestra.run.smithi073.stdout:5683 trunc from 0x4cd26d5 to 0xac5edc7 2024-09-21T14:22:31.074 INFO:teuthology.orchestra.run.smithi073.stdout:5684 read 0x3c16dce thru 0x3c25c45 (0xee78 bytes) 2024-09-21T14:22:31.074 INFO:teuthology.orchestra.run.smithi073.stdout:5686 read 0x60aa184 thru 0x60af76c (0x55e9 bytes) 2024-09-21T14:22:31.074 INFO:teuthology.orchestra.run.smithi073.stdout:5689 punch from 0x1712cf2 to 0x171f191, (0xc49f bytes) 2024-09-21T14:22:31.074 INFO:teuthology.orchestra.run.smithi073.stdout:5692 write 0x8208ecd thru 0x8212ab4 (0x9be8 bytes) 2024-09-21T14:22:31.085 INFO:teuthology.orchestra.run.smithi073.stdout:5693 write 0xb8d6b9d thru 0xb8d766c (0xad0 bytes) 2024-09-21T14:22:31.089 INFO:teuthology.orchestra.run.smithi073.stdout:5694 punch from 0x4293552 to 0x4293a90, (0x53e bytes) 2024-09-21T14:22:31.089 INFO:teuthology.orchestra.run.smithi073.stdout:5695 write 0x123c1fa thru 0x123e419 (0x2220 bytes) 2024-09-21T14:22:31.093 INFO:teuthology.orchestra.run.smithi073.stdout:5696 write 0x63e17b thru 0x641cfb (0x3b81 bytes) 2024-09-21T14:22:31.110 INFO:teuthology.orchestra.run.smithi073.stdout:5697 write 0xaf8b2b9 thru 0xaf8dfa7 (0x2cef bytes) 2024-09-21T14:22:31.114 INFO:teuthology.orchestra.run.smithi073.stdout:5698 trunc from 0xb8d766d to 0xb043ab9 2024-09-21T14:22:31.124 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:31.133 INFO:teuthology.orchestra.run.smithi073.stdout:5701 write 0xe0ab514 thru 0xe0b06f5 (0x51e2 bytes) 2024-09-21T14:22:31.137 INFO:teuthology.orchestra.run.smithi073.stdout:5702 read 0x44cea72 thru 0x44dd536 (0xeac5 bytes) 2024-09-21T14:22:31.137 INFO:teuthology.orchestra.run.smithi073.stdout:5703 write 0xafdd32d thru 0xafed1e1 (0xfeb5 bytes) 2024-09-21T14:22:31.141 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:31.141+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:31.142 INFO:teuthology.orchestra.run.smithi073.stdout:5704 read 0x7eef2cc thru 0x7ef2198 (0x2ecd bytes) 2024-09-21T14:22:31.142 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:31.143 INFO:teuthology.orchestra.run.smithi073.stdout:5706 trunc from 0xe0b06f6 to 0xdd55772 2024-09-21T14:22:31.153 INFO:teuthology.orchestra.run.smithi073.stdout:5710 read 0xb159ad5 thru 0xb166485 (0xc9b1 bytes) 2024-09-21T14:22:31.153 INFO:teuthology.orchestra.run.smithi073.stdout:5712 write 0xdbbf7f8 thru 0xdbc0b38 (0x1341 bytes) 2024-09-21T14:22:31.154 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.154 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.160 INFO:teuthology.orchestra.run.smithi073.stdout:5713 clone 45 order 18 su 8192 sc 15 2024-09-21T14:22:31.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:31.165 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.1. 2024-09-21T14:22:31.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:22:31.242 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:31.242+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:31.343 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:31.343+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:31.387 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.387 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.396 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:31.480 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:31.480+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:31.539 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.539 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.544 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:31.544+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:31.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:31.645 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:31.645+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:31.691 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.691 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.699 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:31.745 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:31.745+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:31.804 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.804 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:31.846 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:31.846+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:31.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:31.946 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:31.947+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:32.025 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.025 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.034 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:32.047 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:32.047+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:32.136 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.136 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.144 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:32.147 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:32.147+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:32.169 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:22:32.248 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:32.248+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:32.251 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.251 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.260 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:32.349 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:32.349+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:32.402 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.402 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:32.450 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:32.450+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:32.551 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:32.551+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:32.554 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.554 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.562 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:32.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:32.652 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:32.652+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:32.705 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:32.705 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:32.711 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:32.711 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:32.713 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:32.728 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:32.744 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:32.744 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:32.757 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:32.758+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:32.760 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:32.776 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:32.776 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:32.792 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:32.792 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:32.808 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:32.808 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:32.818 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386533135000001, 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.819 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.820 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:32.822 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:32.823 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386516493000002, 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:32.825 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.826 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.827 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386491442999997, 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.829 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.831 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386473291000001, 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.832 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:32.834 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.835 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.836 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386450054000001, 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.837 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:32.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.840 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386428987000002, 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.841 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:32.843 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386412626999999, 2024-09-21T14:22:32.845 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:32.847 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:32.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.849 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386382347000001, 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.850 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.851 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:32.852 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.853 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:32.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:32.855 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386367208999999, 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.856 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:32.857 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.858 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.859 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386347612999998, 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.861 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.862 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.863 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:32.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386338836, 2024-09-21T14:22:32.865 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.866 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:32.867 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.868 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.869 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386326474999997, 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.870 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.871 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:32.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:32.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386317058000003, 2024-09-21T14:22:32.874 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:32.876 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.877 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386295867000001, 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.879 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.880 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.881 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.882 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386278759, 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.883 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.884 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.885 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.887 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386261816000001, 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.888 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:32.890 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.891 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386245436999999, 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.892 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.893 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:32.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386204077000002, 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.897 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:32.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:32.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.386184395999997, 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:32.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:32.902 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.903 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.904 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 39.981295949, 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:32.905 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:32.906 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:32.907 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:32.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:32.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:32.910 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:32.911 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:32.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:32.913 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:32.913 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:32.912+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:33.012 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:33.013+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:33.058 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:33.058 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:33.058 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:33.058 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:33.065 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:33.113 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:33.114+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:33.214 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:33.214+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:33.229 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:33.230 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:33.240 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:33.296 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone44 from 0xdd55772 (overlap 0xdf081b) to 0x85d1a9 2024-09-21T14:22:33.314 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:33.314+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:33.415 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:33.415+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:33.425 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:33.425 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827898834999999, 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.426 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:33.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827881321, 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.434 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827864067, 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:33.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827836216999998, 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.439 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827814613000001, 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:33.443 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:33.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827797615000001, 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:33.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827780595, 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.452 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:33.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827763144999999, 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.456 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:33.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:33.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827746007000002, 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.461 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:33.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827722780000002, 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.465 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.467 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:33.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.469 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827706505999998, 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.470 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827682134, 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:33.474 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:33.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.478 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827659566999998, 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.479 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827642646000001, 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.483 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:33.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.487 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827626146999997, 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:33.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.489 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.491 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827605149999997, 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:33.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 63.827583091999998, 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.496 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:33.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 40.653512220000003, 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:33.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.502 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:33.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:33.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.5112790959999991, 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.2785690200000008, 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:33.511 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:33.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:33.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:33.515 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:33.516 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-21T14:22:33.517 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:33.517+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:33.517 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:33.618 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:33.618+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:33.671 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:33.671 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:33.671 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:33.671 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:33.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:33.723 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:33.723+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:33.771 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:33.771 INFO:teuthology.orchestra.run.smithi073.stdout:5716 writesame 0xeee7f8 thru 0xef51d4 data_size 0x1(0x69dd bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5717 read 0x21e1560 thru 0x21e7f34 (0x69d5 bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5719 read 0x25f2fa3 thru 0x25f428d (0x12eb bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5720 punch from 0x6d1e933 to 0x6d22eeb, (0x45b8 bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5721 punch from 0x8fe7c88 to 0x8feac2f, (0x2fa7 bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5724 punch from 0xd922f6c to 0xd92e6c5, (0xb759 bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5725 punch from 0x32fb636 to 0x32fe445, (0x2e0f bytes) 2024-09-21T14:22:33.796 INFO:teuthology.orchestra.run.smithi073.stdout:5726 write 0xfd3996 thru 0xfdfd3a (0xc3a5 bytes) 2024-09-21T14:22:33.818 INFO:teuthology.orchestra.run.smithi073.stdout:5728 write 0xb4aa665 thru 0xb4b279a (0x8136 bytes) 2024-09-21T14:22:33.819 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:33.819+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:33.837 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:33.837 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:33.837 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:33.837 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:33.837 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:33.840 INFO:teuthology.orchestra.run.smithi073.stdout:5729 trunc from 0xdd55772 to 0x3bba3e4 2024-09-21T14:22:33.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:33.870 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:33.876 INFO:teuthology.orchestra.run.smithi073.stdout:5731 writesame 0x465d382 thru 0x46693de data_size 0x43(0xc05d bytes) 2024-09-21T14:22:33.881 INFO:teuthology.orchestra.run.smithi073.stdout:5732 trunc from 0x46693df to 0x370b32 2024-09-21T14:22:33.920 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:33.920+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:33.983 INFO:teuthology.orchestra.run.smithi073.stdout:5733 trunc from 0x370b32 to 0x16cf75b 2024-09-21T14:22:33.999 INFO:teuthology.orchestra.run.smithi073.stdout:5734 write 0x2bb7234 thru 0x2bb8118 (0xee5 bytes) 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.8478800570000002, 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.000 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:22:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:22:34.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:22:34.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84587 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 22020096~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50508273100000001, 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051454017999999997, 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.006 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84587 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494196+0000", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2819999999999999e-06 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494202+0000", 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7919999999999998e-06 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494207+0000", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6249999999999998e-06 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545115+0000", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.050908026000000002 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545182+0000", 2024-09-21T14:22:34.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7642000000000001e-05 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545648+0000", 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046604399999999999 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84589 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50506635799999999, 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052000255000000002, 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84589 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494210+0000", 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494213+0000", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494218+0000", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6489999999999996e-06 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494223+0000", 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6240000000000001e-06 2024-09-21T14:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545674+0000", 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051450666999999999 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545732+0000", 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8224999999999999e-05 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546211+0000", 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000479245 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84592 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50505025000000003, 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052522502999999998, 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.015 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84592 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494229+0000", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494235+0000", 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4969999999999998e-06 2024-09-21T14:22:34.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494240+0000", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4880000000000001e-06 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546237+0000", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051997065000000002 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546297+0000", 2024-09-21T14:22:34.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9769000000000003e-05 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546749+0000", 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0004527 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84601 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25690112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50502925099999996, 2024-09-21T14:22:34.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052709495000000002, 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84601 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494247+0000", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494250+0000", 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2819999999999999e-06 2024-09-21T14:22:34.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494256+0000", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8300000000000001e-06 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494260+0000", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7210000000000002e-06 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546776+0000", 2024-09-21T14:22:34.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052515698 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546835+0000", 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9326000000000001e-05 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546957+0000", 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012202200000000001 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84603 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 26738688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.505012715, 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052901403999999999, 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84603 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494266+0000", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2079999999999999e-06 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494272+0000", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6289999999999998e-06 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494277+0000", 2024-09-21T14:22:34.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8999999999999997e-06 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546980+0000", 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052702950999999998 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547055+0000", 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4831000000000002e-05 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547166+0000", 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011127 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50499635200000004, 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.028 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:34.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:22:34.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50498059399999995, 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:22:34.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:22:34.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50495925500000005, 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:22:34.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:22:34.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:34.041 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50494344700000005, 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.045 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50492717300000001, 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:34.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50490844499999998, 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.051 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:34.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:22:34.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50393900599999997, 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:22:34.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.059 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.503916374, 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:22:34.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.063 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50389973499999996, 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:22:34.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:34.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50388375900000004, 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:22:34.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:34.072 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50386660800000005, 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:22:34.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:22:34.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:22:34.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:22:34.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:22:34.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50384984600000005, 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50364242199999998, 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:34.087 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.50329721400000005, 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:22:34.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.092 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.093 INFO:teuthology.orchestra.run.smithi073.stdout:5735 read 0x16a494a thru 0x16b164e (0xcd05 bytes) 2024-09-21T14:22:34.093 INFO:teuthology.orchestra.run.smithi073.stdout:5736 punch from 0x2545890 to 0x254a94a, (0x50ba bytes) 2024-09-21T14:22:34.094 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:34.095 INFO:teuthology.orchestra.run.smithi073.stdout:5737 writesame 0xdb23d72 thru 0xdb2ba43 data_size 0xd(0x7cd2 bytes) 2024-09-21T14:22:34.095 INFO:teuthology.orchestra.run.smithi073.stdout:5739 read 0xb2fe36f thru 0xb2fe854 (0x4e6 bytes) 2024-09-21T14:22:34.095 INFO:teuthology.orchestra.run.smithi073.stdout:5740 trunc from 0xdb2ba44 to 0x2ea1916 2024-09-21T14:22:34.095 INFO:teuthology.orchestra.run.smithi073.stdout:5742 punch from 0x68e8d4 to 0x69dc55, (0xf381 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5744 read 0x294b69a thru 0x29556b6 (0xa01d bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5745 writesame 0x93f1b1c thru 0x93f87d2 data_size 0x1(0x6cb7 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5746 write 0x8db28b7 thru 0x8dbd762 (0xaeac bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5747 read 0x6751405 thru 0x6757347 (0x5f43 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5748 read 0x626df97 thru 0x62779ca (0x9a34 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5750 read 0x44eeafd thru 0x44fa37f (0xb883 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5752 read 0xdd3643 thru 0xddd886 (0xa244 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5753 read 0x27feeaa thru 0x2807760 (0x88b7 bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5754 writesame 0xbca3f5f thru 0xbca90eb data_size 0x1(0x518d bytes) 2024-09-21T14:22:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:5757 writesame 0x93e8aa8 thru 0x93f8040 data_size 0x1(0xf599 bytes) 2024-09-21T14:22:34.096 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:34.093+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:34.097 INFO:teuthology.orchestra.run.smithi073.stdout:5758 write 0xe767c0 thru 0xe84582 (0xddc3 bytes) 2024-09-21T14:22:34.109 INFO:teuthology.orchestra.run.smithi073.stdout:5759 read 0x3d4f666 thru 0x3d56c9e (0x7639 bytes) 2024-09-21T14:22:34.109 INFO:teuthology.orchestra.run.smithi073.stdout:5760 read 0xb3a297e thru 0xb3b1642 (0xecc5 bytes) 2024-09-21T14:22:34.109 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.109 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.109 INFO:teuthology.orchestra.run.smithi073.stdout:5766 write 0x1cb855e thru 0x1cc4328 (0xbdcb bytes) 2024-09-21T14:22:34.122 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.129 INFO:teuthology.orchestra.run.smithi073.stdout:5768 trunc from 0xbca90ec to 0x533b14f 2024-09-21T14:22:34.147 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.164 INFO:teuthology.orchestra.run.smithi073.stdout:5770 write 0xdb109d9 thru 0xdb12081 (0x16a9 bytes) 2024-09-21T14:22:34.168 INFO:teuthology.orchestra.run.smithi073.stdout:5771 punch from 0xc06dbaa to 0xc07468e, (0x6ae4 bytes) 2024-09-21T14:22:34.168 INFO:teuthology.orchestra.run.smithi073.stdout:5773 punch from 0x978c7f9 to 0x9796d16, (0xa51d bytes) 2024-09-21T14:22:34.168 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.168 INFO:teuthology.orchestra.run.smithi073.stdout:5775 read 0x4f1f28b thru 0x4f261c5 (0x6f3b bytes) 2024-09-21T14:22:34.169 INFO:teuthology.orchestra.run.smithi073.stdout:5776 read 0xc95b5c1 thru 0xc96a221 (0xec61 bytes) 2024-09-21T14:22:34.169 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.169 INFO:teuthology.orchestra.run.smithi073.stdout:5778 write 0x46ea1d4 thru 0x46eeae1 (0x490e bytes) 2024-09-21T14:22:34.174 INFO:teuthology.orchestra.run.smithi073.stdout:5779 read 0x4b99919 thru 0x4b9d6e0 (0x3dc8 bytes) 2024-09-21T14:22:34.174 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:34.174 INFO:teuthology.orchestra.run.smithi073.stdout:5782 write 0x26808ba thru 0x2682e8d (0x25d4 bytes) 2024-09-21T14:22:34.180 INFO:teuthology.orchestra.run.smithi073.stdout:5783 punch from 0xcaeaca2 to 0xcaec10c, (0x146a bytes) 2024-09-21T14:22:34.180 INFO:teuthology.orchestra.run.smithi073.stdout:5784 writesame 0x4af832a thru 0x4b05a73 data_size 0x11(0xd74a bytes) 2024-09-21T14:22:34.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:22:34.193 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:34.194+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:34.198 INFO:teuthology.orchestra.run.smithi073.stdout:5785 read 0x8d48e06 thru 0x8d51988 (0x8b83 bytes) 2024-09-21T14:22:34.198 INFO:teuthology.orchestra.run.smithi073.stdout:5786 write 0x9b30d7f thru 0x9b40883 (0xfb05 bytes) 2024-09-21T14:22:34.240 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.240 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:34.241 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:34.241 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:34.335 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:34.335+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:34.395 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:34.395+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:34.404 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.404 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:34.404 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:34.404 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:34.404 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:34.498 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:34.498+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:34.567 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.345876773999997, 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.854052312, 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.854034607999999, 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.854018936999999, 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.854003423000002, 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853988356000002, 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853973160999999, 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853958055, 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853942945, 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853927579999997, 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853912121, 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853897087999997, 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853881199, 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853855523, 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853840237999997, 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853818273999998, 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.853802291000001, 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.803260903, 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:22:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:22:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:22:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.3392071469999998, 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:22:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:22:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:22:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:22:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.4062961630000004, 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:22:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:22:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:34.658 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:34.660 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:34.660+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:34.760 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:34.760+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:34.802 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.802 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:34.802 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:34.802 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:34.895 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:34.895+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:34.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:34.955 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:34.955 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:34.955 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:34.955 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:34.955 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:34.961 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:34.962+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:34.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:35.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:35.036 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:35.052 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:35.062 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:35.063+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:35.068 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:35.069 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:35.084 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:35.084 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:35.100 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:35.101 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:35.110 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.110 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.117 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:35.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:35.133 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:35.134 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:35.150 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:35.150 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:35.164 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:35.164+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:35.210 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:22:35.213 INFO:teuthology.orchestra.run.smithi073.stdout:5788 read 0x2f301f7 thru 0x2f349ad (0x47b7 bytes) 2024-09-21T14:22:35.216 INFO:teuthology.orchestra.run.smithi073.stdout:5789 trunc from 0xdb12082 to 0xb3a48df 2024-09-21T14:22:35.251 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.251 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:35.265 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:35.265+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:35.366 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:35.366+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:35.402 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.403 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:35.466 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:35.467+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:35.552 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.553 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.562 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:35.606 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:35.606+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:35.666 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.666 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.667 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:35.668+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:35.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:35.768 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:35.769+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:35.816 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.816 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.825 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:35.869 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:35.869+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:35.930 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:35.930 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:35.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:35.969 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:35.970+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:36.071 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:36.071+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:36.079 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.079 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.087 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:36.172 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:36.172+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:36.181 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:36.181 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:36.181 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:22:36.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:22:36.190 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.190 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:36.230 INFO:teuthology.orchestra.run.smithi073.stdout:5790 write 0x8c149e5 thru 0x8c1b542 (0x6b5e bytes) 2024-09-21T14:22:36.237 INFO:teuthology.orchestra.run.smithi073.stdout:5792 punch from 0x9d8e9bb to 0x9d93bb4, (0x51f9 bytes) 2024-09-21T14:22:36.237 INFO:teuthology.orchestra.run.smithi073.stdout:5794 read 0x2284134 thru 0x22844e0 (0x3ad bytes) 2024-09-21T14:22:36.237 INFO:teuthology.orchestra.run.smithi073.stdout:5795 punch from 0x383ff04 to 0x384f523, (0xf61f bytes) 2024-09-21T14:22:36.245 INFO:teuthology.orchestra.run.smithi073.stdout:5796 write 0xcf59ff5 thru 0xcf6375b (0x9767 bytes) 2024-09-21T14:22:36.256 INFO:teuthology.orchestra.run.smithi073.stdout:5797 writesame 0xbbedaf6 thru 0xbbf6c06 data_size 0x1(0x9111 bytes) 2024-09-21T14:22:36.267 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:36.267 INFO:teuthology.orchestra.run.smithi073.stdout:5799 punch from 0xce7e5bb to 0xce7eae5, (0x52a bytes) 2024-09-21T14:22:36.267 INFO:teuthology.orchestra.run.smithi073.stdout:5800 writesame 0xbf8f39f thru 0xbf9a915 data_size 0x5f(0xb577 bytes) 2024-09-21T14:22:36.272 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:36.272+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:36.283 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:36.283 INFO:teuthology.orchestra.run.smithi073.stdout:5802 write 0xb8f7bc1 thru 0xb8f867a (0xaba bytes) 2024-09-21T14:22:36.288 INFO:teuthology.orchestra.run.smithi073.stdout:5803 read 0x13c01e6 thru 0x13c0ffe (0xe19 bytes) 2024-09-21T14:22:36.289 INFO:teuthology.orchestra.run.smithi073.stdout:5804 read 0xba74b51 thru 0xba7870c (0x3bbc bytes) 2024-09-21T14:22:36.289 INFO:teuthology.orchestra.run.smithi073.stdout:5807 punch from 0x3a543e0 to 0x3a560a1, (0x1cc1 bytes) 2024-09-21T14:22:36.289 INFO:teuthology.orchestra.run.smithi073.stdout:5809 writesame 0x96ffeba thru 0x970abe1 data_size 0xc(0xad28 bytes) 2024-09-21T14:22:36.302 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:36.302 INFO:teuthology.orchestra.run.smithi073.stdout:5812 write 0x381bfc6 thru 0x382760c (0xb647 bytes) 2024-09-21T14:22:36.314 INFO:teuthology.orchestra.run.smithi073.stdout:5813 read 0x4c8c64 thru 0x4cdcd1 (0x506e bytes) 2024-09-21T14:22:36.314 INFO:teuthology.orchestra.run.smithi073.stdout:5814 write 0x14b65c7 thru 0x14bcd77 (0x67b1 bytes) 2024-09-21T14:22:36.316 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.316 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.319 INFO:teuthology.orchestra.run.smithi073.stdout:5815 read 0x7c714e4 thru 0x7c74ecd (0x39ea bytes) 2024-09-21T14:22:36.319 INFO:teuthology.orchestra.run.smithi073.stdout:5817 writesame 0x12e14c6 thru 0x12e8d69 data_size 0xe(0x78a4 bytes) 2024-09-21T14:22:36.326 INFO:teuthology.orchestra.run.smithi073.stdout:5819 write 0x915c0c thru 0x921473 (0xb868 bytes) 2024-09-21T14:22:36.328 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:36.337 INFO:teuthology.orchestra.run.smithi073.stdout:5822 punch from 0x3159784 to 0x3164706, (0xaf82 bytes) 2024-09-21T14:22:36.337 INFO:teuthology.orchestra.run.smithi073.stdout:5823 clone 46 order 21 su 131072 sc 13 2024-09-21T14:22:36.373 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:36.373+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:36.431 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.431 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:36.481 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:36.481+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:36.488 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:36.488 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:36.488 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 90, 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:22:36.203841+0000", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:22:31.158403+0000", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5804032, 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:36.489 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:22:36.490 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:36.491 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:36.492 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:36.493 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:36.494 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:36.495 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.9800000190734863, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.9800000190734863, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:36.496 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:36.497 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:36.498 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:36.499 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:36.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:22:36.501 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:36.502 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:36.503 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:36.504 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "86", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 48, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 86, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:36.505 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:36.506 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:36.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.2400000095367432, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.2400000095367432, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.508 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 89, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:22:36.513 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.514 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 82, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:22:36.516 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:36.524 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.525 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.526 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:22:36.527 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:22:36.528 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.529 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:36.530 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.534 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.535 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:22:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.540 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:36.541 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.542 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.544 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:36.546 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:22:36.549 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:36.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.551 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.552 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.554 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:36.555 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "1.1", 2024-09-21T14:22:36.557 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.d", 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: 2 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.e", 2024-09-21T14:22:36.558 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: 7 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:22:36.559 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:22:36.560 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:22:36.561 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:22:36.561 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:22:36.561 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:36.561 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:36.562 INFO:tasks.thrashosds.thrasher:j is {'epoch': 90, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:22:36.203841+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:22:31.158403+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.9800000190734863, 'score_stable': 2.9800000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.1', 'osds': [5, 4]}, {'pgid': '3.d', 'osds': [6, 2]}, {'pgid': '3.e', 'osds': [4, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:22:36.562 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:22:36.581 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.581 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:36.603 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:36.603+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:36.675 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:36.675+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:36.691 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.691 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:36.784 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:36.784+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:36.843 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.843 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:36.852 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:36.875 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:36.876 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:22:36.887 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-21T14:22:36.431825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1733452923,"num_objects":1892,"num_object_clones":480,"num_object_copies":3784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1892,"num_whiteouts":731,"num_read":121973,"num_read_kb":784715,"num_write":17840,"num_write_kb":486280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1460,"num_bytes_recovered":146989630,"num_keys_recovered":292,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14318,"ondisk_log_size":14318,"up":82,"acting":71,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":306424,"kb_used_data":63932,"kb_used_omap":7,"kb_used_meta":242424,"kb_avail":471552776,"statfs":{"total":483183820800,"available":482870042624,"internally_reserved":0,"allocated":65466368,"data_stored":633164304,"data_compressed":36798971,"data_compressed_allocated":43261952,"data_compressed_original":616764544,"omap_allocated":7964,"internal_metadata":248242404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71047206,"num_objects":38,"num_object_clones":3,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-481,"num_objects_misplaced":-32,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":1983,"num_read_kb":169646,"num_write":207,"num_write_kb":17139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":93059183,"num_keys_recovered":164,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000634"},"pg_stats":[{"pgid":"3.1f","version":"84'285","reported_seq":2666,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.167927+0000","last_change":"2024-09-21T14:22:32.168092+0000","last_active":"2024-09-21T14:22:31.169418+0000","last_peered":"2024-09-21T14:22:31.014573+0000","last_clean":"2024-09-21T14:22:31.014573+0000","last_became_active":"2024-09-21T14:21:18.148195+0000","last_became_peered":"2024-09-21T14:21:18.148195+0000","last_unstale":"2024-09-21T14:22:32.167927+0000","last_undegraded":"2024-09-21T14:22:32.167927+0000","last_fullsized":"2024-09-21T14:22:32.167927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:02.385946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":24730350,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3342,"num_read_kb":16890,"num_write":290,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'251","reported_seq":1523,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167246+0000","last_change":"2024-09-21T14:22:29.159013+0000","last_active":"2024-09-21T14:22:31.167246+0000","last_peered":"2024-09-21T14:22:31.167246+0000","last_clean":"2024-09-21T14:22:31.167246+0000","last_became_active":"2024-09-21T14:22:29.158180+0000","last_became_peered":"2024-09-21T14:22:29.158180+0000","last_unstale":"2024-09-21T14:22:31.167246+0000","last_undegraded":"2024-09-21T14:22:31.167246+0000","last_fullsized":"2024-09-21T14:22:31.167246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:47.183599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":60686483,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":1227,"num_read_kb":41673,"num_write":269,"num_write_kb":35066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":32363151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"84'355","reported_seq":2619,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.171909+0000","last_change":"2024-09-21T14:22:32.172104+0000","last_active":"2024-09-21T14:22:31.167957+0000","last_peered":"2024-09-21T14:22:31.111724+0000","last_clean":"2024-09-21T14:22:31.111724+0000","last_became_active":"2024-09-21T14:22:29.164144+0000","last_became_peered":"2024-09-21T14:22:29.164144+0000","last_unstale":"2024-09-21T14:22:32.171909+0000","last_undegraded":"2024-09-21T14:22:32.171909+0000","last_fullsized":"2024-09-21T14:22:32.171909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:24:52.215588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":24876449,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":3460,"num_read_kb":15412,"num_write":387,"num_write_kb":10263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":5698731,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'324","reported_seq":4698,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.173423+0000","last_change":"2024-09-21T14:22:32.173423+0000","last_active":"2024-09-21T14:22:31.168043+0000","last_peered":"2024-09-21T14:22:31.000903+0000","last_clean":"2024-09-21T14:22:31.000903+0000","last_became_active":"2024-09-21T14:21:18.144889+0000","last_became_peered":"2024-09-21T14:21:18.144889+0000","last_unstale":"2024-09-21T14:22:32.173423+0000","last_undegraded":"2024-09-21T14:22:32.173423+0000","last_fullsized":"2024-09-21T14:22:32.173423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:14.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":82404443,"num_objects":78,"num_object_clones":21,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":22,"num_read":4979,"num_read_kb":57330,"num_write":370,"num_write_kb":30372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"83'384","reported_seq":3894,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.170291+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:22:31.170291+0000","last_peered":"2024-09-21T14:22:31.170291+0000","last_clean":"2024-09-21T14:22:31.170291+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:22:31.170291+0000","last_undegraded":"2024-09-21T14:22:31.170291+0000","last_fullsized":"2024-09-21T14:22:31.170291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":88910892,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":24,"num_read":5007,"num_read_kb":39602,"num_write":347,"num_write_kb":18937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"84'398","reported_seq":4068,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.168065+0000","last_change":"2024-09-21T14:22:32.168065+0000","last_active":"2024-09-21T14:22:31.167786+0000","last_peered":"2024-09-21T14:22:31.154005+0000","last_clean":"2024-09-21T14:22:31.154005+0000","last_became_active":"2024-09-21T14:21:18.145349+0000","last_became_peered":"2024-09-21T14:21:18.145349+0000","last_unstale":"2024-09-21T14:22:32.168065+0000","last_undegraded":"2024-09-21T14:22:32.168065+0000","last_fullsized":"2024-09-21T14:22:32.168065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.825572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":75473292,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":5559,"num_read_kb":52992,"num_write":475,"num_write_kb":34242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'700","reported_seq":5155,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.174519+0000","last_change":"2024-09-21T14:22:32.175393+0000","last_active":"2024-09-21T14:22:31.170771+0000","last_peered":"2024-09-21T14:22:30.902007+0000","last_clean":"2024-09-21T14:22:30.902007+0000","last_became_active":"2024-09-21T14:21:18.147680+0000","last_became_peered":"2024-09-21T14:21:18.147680+0000","last_unstale":"2024-09-21T14:22:32.174519+0000","last_undegraded":"2024-09-21T14:22:32.174519+0000","last_fullsized":"2024-09-21T14:22:32.174519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:49:14.560325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":63649759,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":7116,"num_read_kb":29607,"num_write":984,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"84'222","reported_seq":1583,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168485+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:22:31.168485+0000","last_peered":"2024-09-21T14:22:31.168485+0000","last_clean":"2024-09-21T14:22:31.168485+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:22:31.168485+0000","last_undegraded":"2024-09-21T14:22:31.168485+0000","last_fullsized":"2024-09-21T14:22:31.168485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":30008275,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":1439,"num_read_kb":11096,"num_write":212,"num_write_kb":7683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"77'636","reported_seq":2403,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.168084+0000","last_change":"2024-09-21T14:22:32.168257+0000","last_active":"2024-09-21T14:22:31.168465+0000","last_peered":"2024-09-21T14:22:30.629206+0000","last_clean":"2024-09-21T14:22:30.629206+0000","last_became_active":"2024-09-21T14:22:29.161996+0000","last_became_peered":"2024-09-21T14:22:29.161996+0000","last_unstale":"2024-09-21T14:22:32.168084+0000","last_undegraded":"2024-09-21T14:22:32.168084+0000","last_fullsized":"2024-09-21T14:22:32.168084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:51.345803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":19840651,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":4287,"num_read_kb":8801,"num_write":863,"num_write_kb":13875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":2652491,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"84'424","reported_seq":2344,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.170063+0000","last_change":"2024-09-21T14:22:29.162378+0000","last_active":"2024-09-21T14:22:31.170063+0000","last_peered":"2024-09-21T14:22:31.170063+0000","last_clean":"2024-09-21T14:22:31.170063+0000","last_became_active":"2024-09-21T14:22:29.161682+0000","last_became_peered":"2024-09-21T14:22:29.161682+0000","last_unstale":"2024-09-21T14:22:31.170063+0000","last_undegraded":"2024-09-21T14:22:31.170063+0000","last_fullsized":"2024-09-21T14:22:31.170063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:15.765317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":21228159,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":3033,"num_read_kb":11932,"num_write":573,"num_write_kb":6150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":3589748,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"84'490","reported_seq":2261,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168680+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:22:31.168680+0000","last_peered":"2024-09-21T14:22:31.168680+0000","last_clean":"2024-09-21T14:22:31.168680+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:22:31.168680+0000","last_undegraded":"2024-09-21T14:22:31.168680+0000","last_fullsized":"2024-09-21T14:22:31.168680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":66201210,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":24,"num_read":3619,"num_read_kb":23486,"num_write":632,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"84'367","reported_seq":2708,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167993+0000","last_change":"2024-09-21T14:22:29.165906+0000","last_active":"2024-09-21T14:22:31.167993+0000","last_peered":"2024-09-21T14:22:31.167993+0000","last_clean":"2024-09-21T14:22:31.167993+0000","last_became_active":"2024-09-21T14:22:29.164131+0000","last_became_peered":"2024-09-21T14:22:29.164131+0000","last_unstale":"2024-09-21T14:22:31.167993+0000","last_undegraded":"2024-09-21T14:22:31.167993+0000","last_fullsized":"2024-09-21T14:22:31.167993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:06.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":42445262,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":4128,"num_read_kb":15875,"num_write":431,"num_write_kb":9864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":6514761,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T14:22:32.176599+0000","last_change":"2024-09-21T14:22:32.174490+0000","last_active":"2024-09-21T14:22:32.176599+0000","last_peered":"2024-09-21T14:22:32.176599+0000","last_clean":"2024-09-21T14:22:32.176599+0000","last_became_active":"2024-09-21T14:22:32.174175+0000","last_became_peered":"2024-09-21T14:22:32.174175+0000","last_unstale":"2024-09-21T14:22:32.176599+0000","last_undegraded":"2024-09-21T14:22:32.176599+0000","last_fullsized":"2024-09-21T14:22:32.176599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:54.846715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"84'318","reported_seq":2391,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.166416+0000","last_change":"2024-09-21T14:21:52.138940+0000","last_active":"2024-09-21T14:22:31.166416+0000","last_peered":"2024-09-21T14:22:31.166416+0000","last_clean":"2024-09-21T14:22:31.166416+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:22:31.166416+0000","last_undegraded":"2024-09-21T14:22:31.166416+0000","last_fullsized":"2024-09-21T14:22:31.166416+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":55179633,"num_objects":65,"num_object_clones":16,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":25,"num_read":2727,"num_read_kb":11348,"num_write":324,"num_write_kb":5596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.166569+0000","last_change":"2024-09-21T14:22:26.141620+0000","last_active":"2024-09-21T14:22:31.166569+0000","last_peered":"2024-09-21T14:22:31.166569+0000","last_clean":"2024-09-21T14:22:31.166569+0000","last_became_active":"2024-09-21T14:22:26.141314+0000","last_became_peered":"2024-09-21T14:22:26.141314+0000","last_unstale":"2024-09-21T14:22:31.166569+0000","last_undegraded":"2024-09-21T14:22:31.166569+0000","last_fullsized":"2024-09-21T14:22:31.166569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:24:21.823458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"84'424","reported_seq":2602,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.166415+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:22:31.166415+0000","last_peered":"2024-09-21T14:22:31.166415+0000","last_clean":"2024-09-21T14:22:31.166415+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:22:31.166415+0000","last_undegraded":"2024-09-21T14:22:31.166415+0000","last_fullsized":"2024-09-21T14:22:31.166415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":41299182,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":3564,"num_read_kb":14221,"num_write":558,"num_write_kb":4350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.166546+0000","last_change":"2024-09-21T14:22:26.141948+0000","last_active":"2024-09-21T14:22:31.166546+0000","last_peered":"2024-09-21T14:22:31.166546+0000","last_clean":"2024-09-21T14:22:31.166546+0000","last_became_active":"2024-09-21T14:22:26.141693+0000","last_became_peered":"2024-09-21T14:22:26.141693+0000","last_unstale":"2024-09-21T14:22:31.166546+0000","last_undegraded":"2024-09-21T14:22:31.166546+0000","last_fullsized":"2024-09-21T14:22:31.166546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:29:48.520733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"84'294","reported_seq":2566,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.169936+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:22:31.169936+0000","last_peered":"2024-09-21T14:22:31.169936+0000","last_clean":"2024-09-21T14:22:31.169936+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:22:31.169936+0000","last_undegraded":"2024-09-21T14:22:31.169936+0000","last_fullsized":"2024-09-21T14:22:31.169936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":47842671,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":25,"num_read":2446,"num_read_kb":31265,"num_write":283,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":85,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167349+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:22:31.167349+0000","last_peered":"2024-09-21T14:22:31.167349+0000","last_clean":"2024-09-21T14:22:31.167349+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:22:31.167349+0000","last_undegraded":"2024-09-21T14:22:31.167349+0000","last_fullsized":"2024-09-21T14:22:31.167349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"86'391","reported_seq":5041,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T14:22:32.182288+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:22:32.182288+0000","last_peered":"2024-09-21T14:22:32.182288+0000","last_clean":"2024-09-21T14:22:32.182288+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:22:32.182288+0000","last_undegraded":"2024-09-21T14:22:32.182288+0000","last_fullsized":"2024-09-21T14:22:32.182288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":53482924,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":6114,"num_read_kb":30934,"num_write":408,"num_write_kb":7475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"75'251","reported_seq":2356,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167105+0000","last_change":"2024-09-21T14:21:52.139305+0000","last_active":"2024-09-21T14:22:31.167105+0000","last_peered":"2024-09-21T14:22:31.167105+0000","last_clean":"2024-09-21T14:22:31.167105+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:22:31.167105+0000","last_undegraded":"2024-09-21T14:22:31.167105+0000","last_fullsized":"2024-09-21T14:22:31.167105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":38040753,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":19,"num_read":3022,"num_read_kb":21436,"num_write":235,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":187,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168580+0000","last_change":"2024-09-21T14:22:29.157860+0000","last_active":"2024-09-21T14:22:31.168580+0000","last_peered":"2024-09-21T14:22:31.168580+0000","last_clean":"2024-09-21T14:22:31.168580+0000","last_became_active":"2024-09-21T14:22:29.157461+0000","last_became_peered":"2024-09-21T14:22:29.157461+0000","last_unstale":"2024-09-21T14:22:31.168580+0000","last_undegraded":"2024-09-21T14:22:31.168580+0000","last_fullsized":"2024-09-21T14:22:31.168580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:27.251019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":98,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T14:22:32.181750+0000","last_change":"2024-09-21T14:22:32.181750+0000","last_active":"2024-09-21T14:22:32.181750+0000","last_peered":"2024-09-21T14:22:32.181750+0000","last_clean":"2024-09-21T14:22:32.181750+0000","last_became_active":"2024-09-21T14:22:32.177114+0000","last_became_peered":"2024-09-21T14:22:32.177114+0000","last_unstale":"2024-09-21T14:22:32.181750+0000","last_undegraded":"2024-09-21T14:22:32.181750+0000","last_fullsized":"2024-09-21T14:22:32.181750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:00.138436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"84'604","reported_seq":2378,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168403+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:22:31.168403+0000","last_peered":"2024-09-21T14:22:31.168403+0000","last_clean":"2024-09-21T14:22:31.168403+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:22:31.168403+0000","last_undegraded":"2024-09-21T14:22:31.168403+0000","last_fullsized":"2024-09-21T14:22:31.168403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":87014477,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":28,"num_read":4135,"num_read_kb":92381,"num_write":867,"num_write_kb":58385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167842+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:22:31.167842+0000","last_peered":"2024-09-21T14:22:31.167842+0000","last_clean":"2024-09-21T14:22:31.167842+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:22:31.167842+0000","last_undegraded":"2024-09-21T14:22:31.167842+0000","last_fullsized":"2024-09-21T14:22:31.167842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"84'850","reported_seq":1942,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168042+0000","last_change":"2024-09-21T14:21:52.139498+0000","last_active":"2024-09-21T14:22:31.168042+0000","last_peered":"2024-09-21T14:22:31.168042+0000","last_clean":"2024-09-21T14:22:31.168042+0000","last_became_active":"2024-09-21T14:21:52.138744+0000","last_became_peered":"2024-09-21T14:21:52.138744+0000","last_unstale":"2024-09-21T14:22:31.168042+0000","last_undegraded":"2024-09-21T14:22:31.168042+0000","last_fullsized":"2024-09-21T14:22:31.168042+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:16:08.802429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":32811134,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":20,"num_read":3937,"num_read_kb":15459,"num_write":1269,"num_write_kb":7421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2088730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167196+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:22:31.167196+0000","last_peered":"2024-09-21T14:22:31.167196+0000","last_clean":"2024-09-21T14:22:31.167196+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:22:31.167196+0000","last_undegraded":"2024-09-21T14:22:31.167196+0000","last_fullsized":"2024-09-21T14:22:31.167196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'478","reported_seq":2049,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T14:22:32.185206+0000","last_change":"2024-09-21T14:21:52.139320+0000","last_active":"2024-09-21T14:22:32.185206+0000","last_peered":"2024-09-21T14:22:32.185206+0000","last_clean":"2024-09-21T14:22:32.185206+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:22:32.185206+0000","last_undegraded":"2024-09-21T14:22:32.185206+0000","last_fullsized":"2024-09-21T14:22:32.185206+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":80565847,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":2755,"num_read_kb":10952,"num_write":613,"num_write_kb":5335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T14:22:32.173999+0000","last_change":"2024-09-21T14:22:32.172034+0000","last_active":"2024-09-21T14:22:32.173999+0000","last_peered":"2024-09-21T14:22:32.173999+0000","last_clean":"2024-09-21T14:22:32.173999+0000","last_became_active":"2024-09-21T14:22:32.171774+0000","last_became_peered":"2024-09-21T14:22:32.171774+0000","last_unstale":"2024-09-21T14:22:32.173999+0000","last_undegraded":"2024-09-21T14:22:32.173999+0000","last_fullsized":"2024-09-21T14:22:32.173999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:41.624554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"84'526","reported_seq":3179,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.169671+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:22:31.169671+0000","last_peered":"2024-09-21T14:22:31.169671+0000","last_clean":"2024-09-21T14:22:31.169671+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:22:31.169671+0000","last_undegraded":"2024-09-21T14:22:31.169671+0000","last_fullsized":"2024-09-21T14:22:31.169671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":80916051,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":17,"num_read":5396,"num_read_kb":15585,"num_write":629,"num_write_kb":12247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"78'527","reported_seq":3752,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.169764+0000","last_change":"2024-09-21T14:22:29.167445+0000","last_active":"2024-09-21T14:22:31.169764+0000","last_peered":"2024-09-21T14:22:31.169764+0000","last_clean":"2024-09-21T14:22:31.169764+0000","last_became_active":"2024-09-21T14:22:29.166889+0000","last_became_peered":"2024-09-21T14:22:29.166889+0000","last_unstale":"2024-09-21T14:22:31.169764+0000","last_undegraded":"2024-09-21T14:22:31.169764+0000","last_fullsized":"2024-09-21T14:22:31.169764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:48.958859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":134564959,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":21,"num_read":6506,"num_read_kb":53772,"num_write":598,"num_write_kb":55628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":51428954,"num_keys_recovered":47,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"84'333","reported_seq":1180,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.166074+0000","last_change":"2024-09-21T14:22:32.166074+0000","last_active":"2024-09-21T14:22:31.167766+0000","last_peered":"2024-09-21T14:22:30.230899+0000","last_clean":"2024-09-21T14:22:30.230899+0000","last_became_active":"2024-09-21T14:22:29.160078+0000","last_became_peered":"2024-09-21T14:22:29.160078+0000","last_unstale":"2024-09-21T14:22:32.166074+0000","last_undegraded":"2024-09-21T14:22:32.166074+0000","last_fullsized":"2024-09-21T14:22:32.166074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:23:35.484771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":22539148,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":976,"num_read_kb":7672,"num_write":348,"num_write_kb":7164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":5498709,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"84'361","reported_seq":1985,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167090+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:22:31.167090+0000","last_peered":"2024-09-21T14:22:31.167090+0000","last_clean":"2024-09-21T14:22:31.167090+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:22:31.167090+0000","last_undegraded":"2024-09-21T14:22:31.167090+0000","last_fullsized":"2024-09-21T14:22:31.167090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":47225403,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":25,"num_read":2403,"num_read_kb":8471,"num_write":490,"num_write_kb":6576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"84'612","reported_seq":1880,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.168072+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:22:31.168072+0000","last_peered":"2024-09-21T14:22:31.168072+0000","last_clean":"2024-09-21T14:22:31.168072+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:22:31.168072+0000","last_undegraded":"2024-09-21T14:22:31.168072+0000","last_fullsized":"2024-09-21T14:22:31.168072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":48142927,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":3404,"num_read_kb":14179,"num_write":823,"num_write_kb":10714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"78'265","reported_seq":2167,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.173159+0000","last_change":"2024-09-21T14:22:32.173159+0000","last_active":"2024-09-21T14:22:25.543999+0000","last_peered":"2024-09-21T14:22:25.299444+0000","last_clean":"2024-09-21T14:22:25.299444+0000","last_became_active":"2024-09-21T14:21:34.228279+0000","last_became_peered":"2024-09-21T14:21:34.228279+0000","last_unstale":"2024-09-21T14:22:32.173159+0000","last_undegraded":"2024-09-21T14:22:32.173159+0000","last_fullsized":"2024-09-21T14:22:32.173159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:37:39.940306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":43677612,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":2199,"num_read_kb":13334,"num_write":259,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'876","reported_seq":3337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.170006+0000","last_change":"2024-09-21T14:22:23.540379+0000","last_active":"2024-09-21T14:22:31.170006+0000","last_peered":"2024-09-21T14:22:31.170006+0000","last_clean":"2024-09-21T14:22:31.170006+0000","last_became_active":"2024-09-21T14:22:23.539499+0000","last_became_peered":"2024-09-21T14:22:23.539499+0000","last_unstale":"2024-09-21T14:22:31.170006+0000","last_undegraded":"2024-09-21T14:22:31.170006+0000","last_fullsized":"2024-09-21T14:22:31.170006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:00:47.787223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":68084829,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":5883,"num_read_kb":19195,"num_write":1352,"num_write_kb":8696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":7784436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'384","reported_seq":3965,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.171799+0000","last_change":"2024-09-21T14:22:32.172009+0000","last_active":"2024-09-21T14:22:31.168449+0000","last_peered":"2024-09-21T14:22:30.580107+0000","last_clean":"2024-09-21T14:22:30.580107+0000","last_became_active":"2024-09-21T14:21:18.146222+0000","last_became_peered":"2024-09-21T14:21:18.146222+0000","last_unstale":"2024-09-21T14:22:32.171799+0000","last_undegraded":"2024-09-21T14:22:32.171799+0000","last_fullsized":"2024-09-21T14:22:32.171799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:33.613363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":84147051,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":25,"num_read":4748,"num_read_kb":22313,"num_write":382,"num_write_kb":20713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'321","reported_seq":2441,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.168017+0000","last_change":"2024-09-21T14:22:32.168017+0000","last_active":"2024-09-21T14:22:31.168123+0000","last_peered":"2024-09-21T14:22:30.163589+0000","last_clean":"2024-09-21T14:22:30.163589+0000","last_became_active":"2024-09-21T14:21:18.150734+0000","last_became_peered":"2024-09-21T14:21:18.150734+0000","last_unstale":"2024-09-21T14:22:32.168017+0000","last_undegraded":"2024-09-21T14:22:32.168017+0000","last_fullsized":"2024-09-21T14:22:32.168017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:49:59.218864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":47048681,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":2614,"num_read_kb":15183,"num_write":313,"num_write_kb":6939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"84'267","reported_seq":1872,"reported_epoch":86,"state":"remapped+peering","last_fresh":"2024-09-21T14:22:32.170475+0000","last_change":"2024-09-21T14:22:32.170475+0000","last_active":"2024-09-21T14:22:31.166788+0000","last_peered":"2024-09-21T14:22:30.957532+0000","last_clean":"2024-09-21T14:22:30.957532+0000","last_became_active":"2024-09-21T14:22:29.159025+0000","last_became_peered":"2024-09-21T14:22:29.159025+0000","last_unstale":"2024-09-21T14:22:32.170475+0000","last_undegraded":"2024-09-21T14:22:32.170475+0000","last_fullsized":"2024-09-21T14:22:32.170475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:38:50.652435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":58092700,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":1608,"num_read_kb":20459,"num_write":327,"num_write_kb":17871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":13214356,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"72'376","reported_seq":2506,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.166663+0000","last_change":"2024-09-21T14:22:29.158071+0000","last_active":"2024-09-21T14:22:31.166663+0000","last_peered":"2024-09-21T14:22:31.166663+0000","last_clean":"2024-09-21T14:22:31.166663+0000","last_became_active":"2024-09-21T14:22:29.157244+0000","last_became_peered":"2024-09-21T14:22:29.157244+0000","last_unstale":"2024-09-21T14:22:31.166663+0000","last_undegraded":"2024-09-21T14:22:31.166663+0000","last_fullsized":"2024-09-21T14:22:31.166663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:24.971948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":34265035,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3483,"num_read_kb":20651,"num_write":448,"num_write_kb":10656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5431126,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"84'904","reported_seq":3547,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T14:22:31.167912+0000","last_change":"2024-09-21T14:22:29.161103+0000","last_active":"2024-09-21T14:22:31.167912+0000","last_peered":"2024-09-21T14:22:31.167912+0000","last_clean":"2024-09-21T14:22:31.167912+0000","last_became_active":"2024-09-21T14:22:29.160025+0000","last_became_peered":"2024-09-21T14:22:29.160025+0000","last_unstale":"2024-09-21T14:22:31.167912+0000","last_undegraded":"2024-09-21T14:22:31.167912+0000","last_fullsized":"2024-09-21T14:22:31.167912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:16.994801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":27466294,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":6741,"num_read_kb":21109,"num_write":1320,"num_write_kb":12929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":5807026,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1732862536,"num_objects":1888,"num_object_clones":480,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":731,"num_read":121857,"num_read_kb":784615,"num_write":17679,"num_write_kb":483862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1457,"num_bytes_recovered":146399262,"num_keys_recovered":292,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":87924736,"data_stored":905334103,"data_compressed":52458173,"data_compressed_allocated":62152704,"data_compressed_original":883977829,"omap_allocated":0,"internal_metadata":0},"log_size":14198,"ondisk_log_size":14198,"up":64,"acting":53,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98865,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48932,"kb_used_data":9624,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94322908,"statfs":{"total":96636764160,"available":96586657792,"internally_reserved":0,"allocated":9854976,"data_stored":96801885,"data_compressed":5560452,"data_compressed_allocated":6651904,"data_compressed_original":94653341,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.5}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.32700000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.311},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.29799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.35299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49360,"kb_used_data":10052,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94322480,"statfs":{"total":96636764160,"available":96586219520,"internally_reserved":0,"allocated":10293248,"data_stored":92474153,"data_compressed":5576228,"data_compressed_allocated":6381568,"data_compressed_original":89573283,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.379},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38500000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.27600000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56608,"kb_used_data":13204,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94315232,"statfs":{"total":96636764160,"available":96578797568,"internally_reserved":0,"allocated":13520896,"data_stored":137612672,"data_compressed":7665459,"data_compressed_allocated":9670656,"data_compressed_original":134814543,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39300000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.42699999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.7430000000000003,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.40300000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41399999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.316}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64556,"kb_used_data":15392,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94307284,"statfs":{"total":96636764160,"available":96570658816,"internally_reserved":0,"allocated":15761408,"data_stored":145032737,"data_compressed":8798150,"data_compressed_allocated":9916416,"data_compressed_original":140557765,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60199999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.497},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.626}]},{"osd":5,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45700000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.438},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42399999999999999}]}]},{"osd":3,"up_from":0,"seq":55834574868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":0,"seq":55834574868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86968,"kb_used_data":15660,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94284872,"statfs":{"total":96636764160,"available":96547708928,"internally_reserved":0,"allocated":16035840,"data_stored":161242857,"data_compressed":9198682,"data_compressed_allocated":10641408,"data_compressed_original":157165612,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.42499999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.318},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.499}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.52700000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.48299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15028224,"data_stored":160773641,"data_compressed":9198682,"data_compressed_allocated":10641408,"data_compressed_original":157165612,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12337152,"data_stored":119058163,"data_compressed":6888762,"data_compressed_allocated":8253440,"data_compressed_original":115661228,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":140569390,"data_compressed":7948729,"data_compressed_allocated":9515008,"data_compressed_original":137234462,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14757888,"data_stored":144563583,"data_compressed":8798150,"data_compressed_allocated":9916416,"data_compressed_original":140557765,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12443648,"data_stored":136547273,"data_compressed":7633049,"data_compressed_allocated":9625600,"data_compressed_original":134241103,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9240576,"data_stored":91414588,"data_compressed":5542183,"data_compressed_allocated":6336512,"data_compressed_original":88983459,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8855552,"data_stored":96332750,"data_compressed":5560452,"data_compressed_allocated":6651904,"data_compressed_original":94653341,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:22:36.890 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [2, 3, 4, 5] 2024-09-21T14:22:36.890 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '2', '3', '4', '5'] 2024-09-21T14:22:36.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 2 3 4 5 2024-09-21T14:22:36.894 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:36.894+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:36.979 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:36.979+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:36.996 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:36.996 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:37.005 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:37.080 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:37.080+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:37.148 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:37.148 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:37.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:37.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:37.181 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:37.181+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:37.221 INFO:teuthology.orchestra.run.smithi073.stderr:set 3.13 pg_upmap_items mapping to [2->3,4->5] 2024-09-21T14:22:37.231 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:22:37.268 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:37.268 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:37.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:37.320 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:37.320+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:37.324 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone45 from 0xcf6375c (overlap 0x370b32) to 0x2687d6 2024-09-21T14:22:37.333 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:37.333 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:37.353 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:37.353 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:37.382 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:37.382+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:37.385 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:37.385 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:37.402 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:37.419 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:37.419 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:37.435 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:37.435 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:37.440 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:37.441 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:37.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:37.453 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:37.453 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:37.469 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:37.483 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:37.483+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:37.559 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:37.559 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:37.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:37.559 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:37.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4072873480000005, 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:22:37.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:22:37.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84587 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 22020096~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0644900220000002, 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051454017999999997, 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84587 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:37.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494194+0000", 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494196+0000", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2819999999999999e-06 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494202+0000", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7919999999999998e-06 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494207+0000", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6249999999999998e-06 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545115+0000", 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.050908026000000002 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545182+0000", 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7642000000000001e-05 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545648+0000", 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046604399999999999 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84589 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.064473649, 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052000255000000002, 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84589 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494211+0000", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494210+0000", 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494213+0000", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494218+0000", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6489999999999996e-06 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494223+0000", 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6240000000000001e-06 2024-09-21T14:22:37.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545674+0000", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051450666999999999 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.545732+0000", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8224999999999999e-05 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546211+0000", 2024-09-21T14:22:37.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000479245 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84592 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0644575410000003, 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052522502999999998, 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84592 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494227+0000", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494229+0000", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494235+0000", 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4969999999999998e-06 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494240+0000", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4880000000000001e-06 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546237+0000", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.051997065000000002 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546297+0000", 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9769000000000003e-05 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546749+0000", 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0004527 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84601 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25690112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0644365420000002, 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052709495000000002, 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84601 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494248+0000", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494247+0000", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494250+0000", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2819999999999999e-06 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494256+0000", 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8300000000000001e-06 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494260+0000", 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7210000000000002e-06 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546776+0000", 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052515698 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546835+0000", 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9326000000000001e-05 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546957+0000", 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012202200000000001 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.582 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84603 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 26738688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0644200059999998, 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052901403999999999, 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84603 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494264+0000", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494266+0000", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2079999999999999e-06 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494272+0000", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6289999999999998e-06 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494277+0000", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8999999999999997e-06 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.546980+0000", 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052702950999999998 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547055+0000", 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4831000000000002e-05 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547166+0000", 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011127 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0644036430000003, 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:22:37.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:22:37.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0643878850000004, 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:22:37.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:22:37.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:22:37.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.595 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0643665459999996, 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.596 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:22:37.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0643507379999999, 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0643344639999999, 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.604 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:22:37.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:22:37.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0643157360000002, 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:22:37.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.612 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0633462969999998, 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:37.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0633236650000004, 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:22:37.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:22:37.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:22:37.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0633070260000004, 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:22:37.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0632910500000001, 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:37.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0632738990000004, 2024-09-21T14:22:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0632571369999999, 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0630497129999998, 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:22:37.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:22:37.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.0627045050000001, 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:37.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:37.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:37.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:37.646 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:37.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:37.648 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:37.648+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:37.748 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:37.749+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:37.791 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:37.791 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:37.791 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:37.791 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:37.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:37.849 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:37.849+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:37.902 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:37.902 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:37.902 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:37.902 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:37.902 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:37.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:37.951 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:37.952+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:38.050 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:38.049+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.054 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.832673487999998, 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340849026000001, 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:38.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:38.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340831322, 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.064 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340815651, 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.068 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:38.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:38.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340800137000002, 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340785070000003, 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340769874999999, 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.080 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340754769, 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.084 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:38.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:38.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340739659, 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:38.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:38.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:38.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340724293999997, 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:38.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.096 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340708835000001, 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.100 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340693801999997, 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.101 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:38.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340677913, 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:38.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.109 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340652237, 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:38.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340636951999997, 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.114 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:38.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:38.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:38.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340614987999999, 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:38.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.340599005000001, 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:38.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:38.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:38.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.290057617, 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:22:38.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:22:38.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:22:38.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.826003861, 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.132 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:22:38.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.8930928770000008, 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:22:38.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.142 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.143 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:38.151 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:38.152+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:38.219 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:22:38.252 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:38.253+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:38.289 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:38.289 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:38.353 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:38.352+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:38.400 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:38.400 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:38.493 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:38.493+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.160551104999996, 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.160540204, 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.160529796000006, 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.160521854999999, 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083783870000005, 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083765459999995, 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:38.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083739711000007, 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:38.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:38.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.581 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083718681999997, 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.582 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:38.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:38.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083692831999997, 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.586 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083674353000006, 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.590 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:38.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083656231000006, 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.594 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083635869000005, 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083616176999996, 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083599199000005, 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083579526999998, 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.610 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:38.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083550063999994, 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:38.614 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.618 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083507818000001, 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.622 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083488829999993, 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.626 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083465472, 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:38.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:38.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.630 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.083442118999997, 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:38.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:38.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:38.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:38.635 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:38.635 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.636 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:38.636 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:38.636+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:38.736 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:38.737+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:38.779 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:38.779 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:38.837 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:38.837+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:38.930 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:38.930 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:38.930 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:38.930 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:38.930 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:38.937 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:38.938+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:38.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:39.038 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:39.038+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.498816942000005, 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653240580000002, 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653226547999999, 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.054 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653210549000001, 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653195652999997, 2024-09-21T14:22:39.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653180716999998, 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.062 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:39.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.066 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653165737000002, 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:39.070 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653151121000001, 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:39.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:39.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653136074000003, 2024-09-21T14:22:39.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:39.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:39.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653121515000002, 2024-09-21T14:22:39.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653106790000002, 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:39.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.085 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653092350000001, 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:39.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653078000000001, 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.089 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:39.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:39.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653057934000003, 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:39.093 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.097 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653043023999999, 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:39.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653028083999999, 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.653008391, 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.109 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.005527502, 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:22:39.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:22:39.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:22:39.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.895378319000001, 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:22:39.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.991815774999999, 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:39.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:39.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:39.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:39.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:39.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:39.125 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:39.126 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:39.170 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:39.170+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:39.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:39.239+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:39.243 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:39.243 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:39.243 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:39.243 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:39.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:39.340 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:39.340+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:39.412 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:39.412 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:39.412 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:39.413 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:39.413 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:39.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:39.441 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:39.441+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:39.443 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-21T14:22:39.459 INFO:teuthology.orchestra.run.smithi073.stdout:5825 read 0x3a06262 thru 0x3a12db0 (0xcb4f bytes) 2024-09-21T14:22:39.460 INFO:teuthology.orchestra.run.smithi073.stdout:5826 write 0x2c33afe thru 0x2c42838 (0xed3b bytes) 2024-09-21T14:22:39.472 INFO:teuthology.orchestra.run.smithi073.stdout:5827 punch from 0x5fb8b57 to 0x5fbb54c, (0x29f5 bytes) 2024-09-21T14:22:39.472 INFO:teuthology.orchestra.run.smithi073.stdout:5829 writesame 0x8dcb2e8 thru 0x8dd310e data_size 0xf(0x7e27 bytes) 2024-09-21T14:22:39.479 INFO:teuthology.orchestra.run.smithi073.stdout:5830 read 0x66b5282 thru 0x66bdce8 (0x8a67 bytes) 2024-09-21T14:22:39.479 INFO:teuthology.orchestra.run.smithi073.stdout:5831 read 0x62e5454 thru 0x62f2e2f (0xd9dc bytes) 2024-09-21T14:22:39.479 INFO:teuthology.orchestra.run.smithi073.stdout:5832 write 0x367b3f9 thru 0x367ba46 (0x64e bytes) 2024-09-21T14:22:39.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:39.490 INFO:teuthology.orchestra.run.smithi073.stdout:5833 trunc from 0xcf6375c to 0x91a9bb6 2024-09-21T14:22:39.525 INFO:teuthology.orchestra.run.smithi073.stdout:5834 write 0x2d388b thru 0x2dd331 (0x9aa7 bytes) 2024-09-21T14:22:39.536 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:39.537 INFO:teuthology.orchestra.run.smithi073.stdout:5837 punch from 0x7a3e6e2 to 0x7a4025b, (0x1b79 bytes) 2024-09-21T14:22:39.537 INFO:teuthology.orchestra.run.smithi073.stdout:5839 read 0x178d6d9 thru 0x1798be2 (0xb50a bytes) 2024-09-21T14:22:39.542 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:39.542+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:39.546 INFO:teuthology.orchestra.run.smithi073.stdout:5840 write 0xc3661ea thru 0xc370554 (0xa36b bytes) 2024-09-21T14:22:39.549 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:39.550 INFO:teuthology.orchestra.run.smithi073.stdout:5843 write 0x7f09c2c thru 0x7f0d563 (0x3938 bytes) 2024-09-21T14:22:39.554 INFO:teuthology.orchestra.run.smithi073.stdout:5844 punch from 0x92e7b42 to 0x92f11e2, (0x96a0 bytes) 2024-09-21T14:22:39.566 INFO:teuthology.orchestra.run.smithi073.stdout:5847 trunc from 0xc370555 to 0x2b01aeb 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.830375376999999, 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.599 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:22:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.830298867, 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.604 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:22:39.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.608 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.829488903000001, 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:22:39.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.829461846000001, 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.613 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.829013623000002, 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828991283000001, 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:22:39.622 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:39.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:22:39.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:22:39.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828949592000001, 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:39.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:22:39.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828790927, 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.631 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:39.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.635 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828758506, 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.636 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:39.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:22:39.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828729976999998, 2024-09-21T14:22:39.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:22:39.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:22:39.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828708881000001, 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:22:39.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:22:39.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828676711, 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.654 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828600221999999, 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828577282000001, 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:22:39.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:22:39.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828560265, 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:22:39.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828411412000001, 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:22:39.668 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.672 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828395702000002, 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.673 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:39.677 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828380279000001, 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:22:39.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:22:39.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:22:39.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828248213999998, 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.828213157, 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:39.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:39.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:39.691 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:39.692 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:39.692 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:39.693 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:5848 read 0x236a464 thru 0x237661f (0xc1bc bytes) 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:5850 write 0x77c0b9d thru 0x77c4cf7 (0x415b bytes) 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:5851 read 0x6ce8ac3 thru 0x6cefe4e (0x738c bytes) 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:5852 read 0x59f2504 thru 0x59feefa (0xc9f7 bytes) 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:5853 write 0x9b2554 thru 0x9b5223 (0x2cd0 bytes) 2024-09-21T14:22:39.694 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:39.695 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:39.695 INFO:teuthology.orchestra.run.smithi073.stdout:5854 write 0xe65db7 thru 0xe71717 (0xb961 bytes) 2024-09-21T14:22:39.695 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:39.695 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:5855 write 0xe0d1ad1 thru 0xe0d9fad (0x84dd bytes) 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:5857 punch from 0x9a6423a to 0x9a74202, (0xffc8 bytes) 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:5858 write 0x399e5e7 thru 0x39aab98 (0xc5b2 bytes) 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:5859 write 0x9781d4b thru 0x9784070 (0x2326 bytes) 2024-09-21T14:22:39.696 INFO:teuthology.orchestra.run.smithi073.stdout:5860 trunc from 0xe0d9fae to 0x9084d3a 2024-09-21T14:22:39.697 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:39.692+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:39.701 INFO:teuthology.orchestra.run.smithi073.stdout:5861 writesame 0x248830e thru 0x2491711 data_size 0x2(0x9404 bytes) 2024-09-21T14:22:39.724 INFO:teuthology.orchestra.run.smithi073.stdout:5862 write 0xdb11269 thru 0xdb1a66d (0x9405 bytes) 2024-09-21T14:22:39.729 INFO:teuthology.orchestra.run.smithi073.stdout:5863 punch from 0x9b1e4d4 to 0x9b2978d, (0xb2b9 bytes) 2024-09-21T14:22:39.729 INFO:teuthology.orchestra.run.smithi073.stdout:5865 write 0xaaae32b thru 0xaab43d4 (0x60aa bytes) 2024-09-21T14:22:39.732 INFO:teuthology.orchestra.run.smithi073.stdout:5866 write 0x9fa10cc thru 0x9fa933a (0x826f bytes) 2024-09-21T14:22:39.735 INFO:teuthology.orchestra.run.smithi073.stdout:5867 clone 47 order 20 su 16384 sc 12 2024-09-21T14:22:39.793 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:39.793+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:39.801 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:39.801 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:39.809 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:39.874 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:39.875 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:39.890 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:39.891 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:39.894 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:39.894+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:39.907 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:39.908 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:39.913 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:39.913 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:39.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:39.923 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:39.940 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:39.940 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:39.995 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:39.995+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:40.065 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.065 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:40.118 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:40.118+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:40.176 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.176 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:40.196 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:40.196+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:40.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:22:40.297 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:40.297+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.335 INFO:teuthology.orchestra.run.smithi073.stdout:truncating image image_client.0-clone46 from 0xdb1a66e (overlap 0x2b01aeb) to 0x1580d75 2024-09-21T14:22:40.346 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:40.398 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:40.398+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:40.448 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.448 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:40.500 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:40.500+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:40.599 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:40.599+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:40.600 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.600 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:40.699 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:40.700+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:40.751 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.751 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:40.801 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:40.801+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:40.902 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:40.902+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:40.902 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:40.902 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:40.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:41.003 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:41.003+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:41.014 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.014 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:41.103 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:41.103+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:41.125 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.125 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:41.205 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:41.205+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:41.248 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:22:41.291 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.292 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:41.305 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:41.306+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:41.406 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:41.406+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:41.444 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-21T14:22:41.451 INFO:teuthology.orchestra.run.smithi073.stdout:5868 write 0x112787 thru 0x11d772 (0xafec bytes) 2024-09-21T14:22:41.458 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.459 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:41.486 INFO:teuthology.orchestra.run.smithi073.stdout:5869 trunc from 0xdb1a66e to 0x407dd78 2024-09-21T14:22:41.506 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:41.507+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:41.514 INFO:teuthology.orchestra.run.smithi073.stdout:5870 write 0x4536888 thru 0x4538860 (0x1fd9 bytes) 2024-09-21T14:22:41.515 INFO:teuthology.orchestra.run.smithi073.stdout:5872 writesame 0x2e8eff9 thru 0x2e9b4b9 data_size 0x13(0xc4c1 bytes) 2024-09-21T14:22:41.531 INFO:teuthology.orchestra.run.smithi073.stdout:5873 punch from 0xe88c98 to 0xe982a7, (0xf60f bytes) 2024-09-21T14:22:41.540 INFO:teuthology.orchestra.run.smithi073.stdout:5874 trunc from 0x4538861 to 0xc7c7e1c 2024-09-21T14:22:41.546 INFO:teuthology.orchestra.run.smithi073.stdout:5875 writesame 0x9b63606 thru 0x9b7172d data_size 0x83(0xe128 bytes) 2024-09-21T14:22:41.554 INFO:teuthology.orchestra.run.smithi073.stdout:5876 writesame 0xeeb611 thru 0xef2422 data_size 0x2(0x6e12 bytes) 2024-09-21T14:22:41.569 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.569 INFO:teuthology.orchestra.run.smithi073.stdout:5878 read 0x978a2fc thru 0x978f820 (0x5525 bytes) 2024-09-21T14:22:41.569 INFO:teuthology.orchestra.run.smithi073.stdout:5879 read 0xc246678 thru 0xc250ca4 (0xa62d bytes) 2024-09-21T14:22:41.576 INFO:teuthology.orchestra.run.smithi073.stdout:5880 write 0xd84468c thru 0xd849496 (0x4e0b bytes) 2024-09-21T14:22:41.585 INFO:teuthology.orchestra.run.smithi073.stdout:5881 write 0xddbcf3b thru 0xddbe97a (0x1a40 bytes) 2024-09-21T14:22:41.593 INFO:teuthology.orchestra.run.smithi073.stdout:5883 writesame 0xe67b394 thru 0xe68143c data_size 0x31(0x60a9 bytes) 2024-09-21T14:22:41.593 INFO:teuthology.orchestra.run.smithi073.stdout:5884 write 0xe21f6 thru 0xf20bd (0xfec8 bytes) 2024-09-21T14:22:41.607 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.607 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.608 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:41.608+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:41.608 INFO:teuthology.orchestra.run.smithi073.stdout:5885 trunc from 0xe68143d to 0xa2bdf16 2024-09-21T14:22:41.617 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:41.617 INFO:teuthology.orchestra.run.smithi073.stdout:5886 punch from 0x17692bf to 0x1777631, (0xe372 bytes) 2024-09-21T14:22:41.627 INFO:teuthology.orchestra.run.smithi073.stdout:5887 trunc from 0xa2bdf16 to 0x1ee72da 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:5888 writesame 0x7435588 thru 0x743c952 data_size 0x3(0x73cb bytes) 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:5889 read 0x48673b0 thru 0x48708af (0x9500 bytes) 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:5890 punch from 0x247bfd to 0x24f06b, (0x746e bytes) 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:5892 read 0x117dbff thru 0x1180f46 (0x3348 bytes) 2024-09-21T14:22:41.691 INFO:teuthology.orchestra.run.smithi073.stdout:5894 writesame 0x21662cf thru 0x2170531 data_size 0x5d(0xa263 bytes) 2024-09-21T14:22:41.692 INFO:teuthology.orchestra.run.smithi073.stdout:5895 read 0x25db07a thru 0x25e4fc7 (0x9f4e bytes) 2024-09-21T14:22:41.697 INFO:teuthology.orchestra.run.smithi073.stdout:5896 trunc from 0x743c953 to 0xb5b2d5d 2024-09-21T14:22:41.708 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:41.709+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:41.709 INFO:teuthology.orchestra.run.smithi073.stdout:5898 trunc from 0xb5b2d5d to 0x5c82ff0 2024-09-21T14:22:41.714 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.722 INFO:teuthology.orchestra.run.smithi073.stdout:5901 trunc from 0x5c82ff0 to 0xc87dfdc 2024-09-21T14:22:41.727 INFO:teuthology.orchestra.run.smithi073.stdout:5903 write 0xc10dac2 thru 0xc11a0a1 (0xc5e0 bytes) 2024-09-21T14:22:41.735 INFO:teuthology.orchestra.run.smithi073.stdout:5904 read 0x941a30f thru 0x94207ea (0x64dc bytes) 2024-09-21T14:22:41.743 INFO:teuthology.orchestra.run.smithi073.stdout:5905 writesame 0xe6be94c thru 0xe6cdaa0 data_size 0x1(0xf155 bytes) 2024-09-21T14:22:41.743 INFO:teuthology.orchestra.run.smithi073.stdout:5906 read 0x299b1e1 thru 0x299d60c (0x242c bytes) 2024-09-21T14:22:41.743 INFO:teuthology.orchestra.run.smithi073.stdout:5907 writesame 0x5262120 thru 0x526ebc9 data_size 0x3(0xcaaa bytes) 2024-09-21T14:22:41.743 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.744 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.754 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:41.756 INFO:teuthology.orchestra.run.smithi073.stdout:5908 punch from 0x7ad7483 to 0x7ad9aa4, (0x2621 bytes) 2024-09-21T14:22:41.756 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.756 INFO:teuthology.orchestra.run.smithi073.stdout:5912 punch from 0xafa379d to 0xafa8f9d, (0x5800 bytes) 2024-09-21T14:22:41.756 INFO:teuthology.orchestra.run.smithi073.stdout:5913 read 0x4dea41f thru 0x4df3956 (0x9538 bytes) 2024-09-21T14:22:41.757 INFO:teuthology.orchestra.run.smithi073.stdout:5914 writesame 0xa0813c0 thru 0xa09128f data_size 0x30(0xfed0 bytes) 2024-09-21T14:22:41.764 INFO:teuthology.orchestra.run.smithi073.stdout:5915 write 0x77566bb thru 0x775e6f5 (0x803b bytes) 2024-09-21T14:22:41.768 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.768 INFO:teuthology.orchestra.run.smithi073.stdout:5918 write 0x8ee8eaf thru 0x8ef241e (0x9570 bytes) 2024-09-21T14:22:41.774 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.774 INFO:teuthology.orchestra.run.smithi073.stdout:5920 write 0x5bbdcd8 thru 0x5bc3fd6 (0x62ff bytes) 2024-09-21T14:22:41.778 INFO:teuthology.orchestra.run.smithi073.stdout:5921 writesame 0x260af30 thru 0x260d3ac data_size 0x1(0x247d bytes) 2024-09-21T14:22:41.790 INFO:teuthology.orchestra.run.smithi073.stdout:5922 trunc from 0xe6cdaa1 to 0x9b0323c 2024-09-21T14:22:41.810 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:41.810+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:41.811 INFO:teuthology.orchestra.run.smithi073.stdout:5923 write 0xde49782 thru 0xde57698 (0xdf17 bytes) 2024-09-21T14:22:41.819 INFO:teuthology.orchestra.run.smithi073.stdout:5924 write 0xb25d13b thru 0xb25d6ce (0x594 bytes) 2024-09-21T14:22:41.822 INFO:teuthology.orchestra.run.smithi073.stdout:5925 read 0x46f7ea thru 0x470641 (0xe58 bytes) 2024-09-21T14:22:41.823 INFO:teuthology.orchestra.run.smithi073.stdout:5927 writesame 0xd39bd5e thru 0xd3aa030 data_size 0x1(0xe2d3 bytes) 2024-09-21T14:22:41.836 INFO:teuthology.orchestra.run.smithi073.stdout:5930 punch from 0x206749a to 0x206caf2, (0x5658 bytes) 2024-09-21T14:22:41.836 INFO:teuthology.orchestra.run.smithi073.stdout:5931 read 0x151e0da thru 0x15209bf (0x28e6 bytes) 2024-09-21T14:22:41.837 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.847 INFO:teuthology.orchestra.run.smithi073.stdout:5933 trunc from 0xde57699 to 0x534011a 2024-09-21T14:22:41.867 INFO:teuthology.orchestra.run.smithi073.stdout:5934 write 0x58133bc thru 0x58231a4 (0xfde9 bytes) 2024-09-21T14:22:41.871 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:41.871 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:5936 writesame 0x3110fd3 thru 0x3115d64 data_size 0x1(0x4d92 bytes) 2024-09-21T14:22:41.880 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:41.898 INFO:teuthology.orchestra.run.smithi073.stdout:5939 writesame 0xd01385f thru 0xd01e589 data_size 0x7(0xad2b bytes) 2024-09-21T14:22:41.898 INFO:teuthology.orchestra.run.smithi073.stdout:5940 read 0xae5647d thru 0xae61f5a (0xbade bytes) 2024-09-21T14:22:41.898 INFO:teuthology.orchestra.run.smithi073.stdout:5941 writesame 0x249095 thru 0x24e34a data_size 0x3(0x52b6 bytes) 2024-09-21T14:22:41.911 INFO:teuthology.orchestra.run.smithi073.stdout:5942 read 0x466c724 thru 0x467ba21 (0xf2fe bytes) 2024-09-21T14:22:41.911 INFO:teuthology.orchestra.run.smithi073.stdout:5944 read 0x2ef232d thru 0x2ef2c4c (0x920 bytes) 2024-09-21T14:22:41.911 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.911 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.911 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:41.911+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:41.921 INFO:teuthology.orchestra.run.smithi073.stdout:5947 trunc from 0xd01e58a to 0x3d8c7d2 2024-09-21T14:22:41.932 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:41.945 INFO:teuthology.orchestra.run.smithi073.stdout:5949 write 0x9baf2c1 thru 0x9bba211 (0xaf51 bytes) 2024-09-21T14:22:41.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:41.954 INFO:teuthology.orchestra.run.smithi073.stdout:5950 write 0x910b36d thru 0x91192b0 (0xdf44 bytes) 2024-09-21T14:22:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:5951 read 0x7441613 thru 0x7450679 (0xf067 bytes) 2024-09-21T14:22:41.962 INFO:teuthology.orchestra.run.smithi073.stdout:5952 write 0x6e3c22f thru 0x6e473c5 (0xb197 bytes) 2024-09-21T14:22:41.980 INFO:teuthology.orchestra.run.smithi073.stdout:5953 trunc from 0x9bba212 to 0x189359 2024-09-21T14:22:42.011 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:42.011+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:42.016 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:42.016 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:42.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:42.045 INFO:teuthology.orchestra.run.smithi073.stdout:5954 writesame 0xd85d9f4 thru 0xd864d82 data_size 0x39(0x738f bytes) 2024-09-21T14:22:42.045 INFO:teuthology.orchestra.run.smithi073.stdout:5955 writesame 0xcf757a1 thru 0xcf7f473 data_size 0x1(0x9cd3 bytes) 2024-09-21T14:22:42.061 INFO:teuthology.orchestra.run.smithi073.stdout:5957 trunc from 0xd864d83 to 0xc3b9912 2024-09-21T14:22:42.072 INFO:teuthology.orchestra.run.smithi073.stdout:5958 write 0x7a78bba thru 0x7a8629b (0xd6e2 bytes) 2024-09-21T14:22:42.079 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:42.079 INFO:teuthology.orchestra.run.smithi073.stdout:5960 read 0x9a9fc12 thru 0x9aac967 (0xcd56 bytes) 2024-09-21T14:22:42.079 INFO:teuthology.orchestra.run.smithi073.stdout:5961 read 0x260d9d3 thru 0x2618c39 (0xb267 bytes) 2024-09-21T14:22:42.080 INFO:teuthology.orchestra.run.smithi073.stdout:5963 write 0x966afc1 thru 0x96752f9 (0xa339 bytes) 2024-09-21T14:22:42.086 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:42.086 INFO:teuthology.orchestra.run.smithi073.stdout:5966 punch from 0x25dae78 to 0x25dbd4d, (0xed5 bytes) 2024-09-21T14:22:42.086 INFO:teuthology.orchestra.run.smithi073.stdout:5967 write 0x33c91cf thru 0x33cec3b (0x5a6d bytes) 2024-09-21T14:22:42.090 INFO:teuthology.orchestra.run.smithi073.stdout:5968 trunc from 0xc3b9912 to 0xbf5b02a 2024-09-21T14:22:42.094 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:42.094 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:42.095 INFO:teuthology.orchestra.run.smithi073.stdout:5971 writesame 0x246dc0b thru 0x2477640 data_size 0x1(0x9a36 bytes) 2024-09-21T14:22:42.109 INFO:teuthology.orchestra.run.smithi073.stdout:5972 read 0x8fe21bc thru 0x8fe4050 (0x1e95 bytes) 2024-09-21T14:22:42.112 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:42.113+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:42.113 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:42.114 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:42.115 INFO:teuthology.orchestra.run.smithi073.stdout:5973 write 0xc28c33c thru 0xc29bfdd (0xfca2 bytes) 2024-09-21T14:22:42.120 INFO:teuthology.orchestra.run.smithi073.stdout:5974 read 0x8edef9 thru 0x8f55c7 (0x76cf bytes) 2024-09-21T14:22:42.121 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:42.121 INFO:teuthology.orchestra.run.smithi073.stdout:5979 read 0x6e627ac thru 0x6e70e64 (0xe6b9 bytes) 2024-09-21T14:22:42.121 INFO:teuthology.orchestra.run.smithi073.stdout:5980 write 0x56782b6 thru 0x567f8f5 (0x7640 bytes) 2024-09-21T14:22:42.127 INFO:teuthology.orchestra.run.smithi073.stdout:5981 write 0x6972150 thru 0x69799a0 (0x7851 bytes) 2024-09-21T14:22:42.134 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:42.134 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:42.134 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:42.134 INFO:teuthology.orchestra.run.smithi073.stdout:5983 write 0x83b8fe7 thru 0x83c6f1e (0xdf38 bytes) 2024-09-21T14:22:42.139 INFO:teuthology.orchestra.run.smithi073.stdout:5984 writesame 0x433f933 thru 0x4348e02 data_size 0x8(0x94d0 bytes) 2024-09-21T14:22:42.147 INFO:teuthology.orchestra.run.smithi073.stdout:5988 punch from 0x1129d0 to 0x11b92b, (0x8f5b bytes) 2024-09-21T14:22:42.147 INFO:teuthology.orchestra.run.smithi073.stdout:5989 read 0x323cbaa thru 0x3240ef9 (0x4350 bytes) 2024-09-21T14:22:42.147 INFO:teuthology.orchestra.run.smithi073.stdout:5990 writesame 0xb789c1e thru 0xb7900a3 data_size 0x3(0x6486 bytes) 2024-09-21T14:22:42.153 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:42.153 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:42.154 INFO:teuthology.orchestra.run.smithi073.stdout:5993 read 0x90b1bca thru 0x90b2087 (0x4be bytes) 2024-09-21T14:22:42.162 INFO:teuthology.orchestra.run.smithi073.stdout:5995 trunc from 0xc29bfde to 0x51bf42a 2024-09-21T14:22:42.165 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:42.165 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:42.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:42.165 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:42.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.772042217000006, 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:42.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.772031315999996, 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:42.174 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.772020908000002, 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:42.178 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.772012966999995, 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.182 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695274982000001, 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695256572000005, 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695230823000003, 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695209793999993, 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695183944000007, 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695165465000002, 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695147343000002, 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695126981000001, 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695107289000006, 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695090311000001, 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695070638999994, 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.695041176000004, 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.694998929999997, 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.694979942000003, 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.694956583999996, 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.694933230999993, 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.248 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:42.248 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:42.248 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:42.248 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T14:22:42.248 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:22:42.248 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:22:42.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T14:22:42.249 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:42.250 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:42.249+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:42.250 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:42.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:42.251 INFO:teuthology.orchestra.run.smithi073.stdout:5996 read 0x381c1b6 thru 0x381d3d6 (0x1221 bytes) 2024-09-21T14:22:42.252 INFO:teuthology.orchestra.run.smithi073.stdout:5997 write 0x4de8748 thru 0x4df665c (0xdf15 bytes) 2024-09-21T14:22:42.252 INFO:teuthology.orchestra.run.smithi073.stdout:compare and write disabled 2024-09-21T14:22:42.252 INFO:teuthology.orchestra.run.smithi073.stdout:5999 write 0x45c826e thru 0x45ca6c5 (0x2458 bytes) 2024-09-21T14:22:42.252 INFO:teuthology.orchestra.run.smithi073.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-21T14:22:42.349 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:42.349+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:42.355 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:22:42.355 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:42.390 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:42.391 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:42.400 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:42.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:42.400 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:42.400 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:42.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:42.410 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:42.410 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:42.428 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:42.428 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:42.481 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:42.481+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:42.534 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:42.534 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:42.534 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:42.534 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:42.534 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:42.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:42.584 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:42.584+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:42.652 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:42.652+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:42.686 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:42.686 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.14425344, 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:42.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:42.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:42.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298677077999997, 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:42.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298663046000001, 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:42.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:42.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298647047000003, 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298632151, 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:42.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:42.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:42.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298617215, 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:42.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298602234999997, 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:42.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:42.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298587619000003, 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298572571999998, 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:42.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298558012999997, 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298543287999998, 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298528847999997, 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298514498000003, 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298494431999998, 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298479522000001, 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298464582000001, 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.298444889000002, 2024-09-21T14:22:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.650963999999998, 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.540814817000001, 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.637252273000001, 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:22:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:22:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:22:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:42.772 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:42.773 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:42.774 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:42.773+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:42.874 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:42.874+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:42.877 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:42.877 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:42.885 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:42.975 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:42.976+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:42.988 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:42.988 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:42.988 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:42.988 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:42.988 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:42.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:43.083 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:43.083+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:43.142 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:43.142 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.374547014000001, 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:22:43.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:43.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:43.147 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.374470504000001, 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:22:43.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:22:43.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:22:43.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.373660539999999, 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:22:43.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:22:43.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.373633482999999, 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.37318526, 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:43.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:22:43.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:22:43.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:22:43.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.373162919999999, 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.165 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:22:43.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.169 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.373121228999999, 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:22:43.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:22:43.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372962564000002, 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:43.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372930143000001, 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:22:43.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372901614, 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:22:43.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:22:43.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:22:43.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372880517999999, 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:22:43.187 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:43.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:22:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.191 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372848348000002, 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372771859, 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:43.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:22:43.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372748918999999, 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:22:43.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372731902000002, 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:43.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372583048999999, 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:43.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:22:43.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372567339, 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.213 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:22:43.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:22:43.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372551915999999, 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:22:43.222 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372419851, 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.223 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:22:43.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.372384793999998, 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.232 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:43.233 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:43.234 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:43.233+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:43.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:22:43.334 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:43.334+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:43.389 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:43.390 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:43.399 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:43.440 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:43.440+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:43.536 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:43.537+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:43.540 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:43.540 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:43.541 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:43.541 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:43.541 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:43.550 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:43.638 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:43.638+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 19.691492351000001, 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.194872531, 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:22:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:22:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.194846961, 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:22:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.193892572999999, 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:22:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:22:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.193578071999999, 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:22:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.192162818, 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:22:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:22:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:22:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.191115368, 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:22:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.189831954000001, 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:22:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:22:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.188510728000001, 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:22:43.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.187469794, 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.743 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.187218354000001, 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.744 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:22:43.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.186172604999999, 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.749 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:22:43.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.753 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.185785663000001, 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.754 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.184850079, 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:43.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:22:43.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.18475741, 2024-09-21T14:22:43.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:22:43.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:22:43.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.180676192, 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.18049336, 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:22:43.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:22:43.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:22:43.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.180169648, 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.778 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:43.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:22:43.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.180048194999999, 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.783 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:22:43.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:22:43.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:22:43.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:43.788 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 10.178789596, 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:43.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:43.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:22:43.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:43.793 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:43.794 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:43.795 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:43.794+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:43.895 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:43.895+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:43.898 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:43.898 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:43.898 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:43.898 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:43.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:43.996 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:43.996+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:44.009 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:44.009 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:44.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:44.101 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:44.102+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:44.162 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:44.162 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730380695999997, 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:44.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:44.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.167 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730364054000006, 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:44.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730339004000001, 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.172 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:44.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.176 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730320852000006, 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.177 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:44.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:44.181 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730297614999998, 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:44.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730276548000006, 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:44.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730260188000003, 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:44.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730229907999998, 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:44.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730214770000003, 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730195174000002, 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.203 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.207 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730186396999997, 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:44.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.211 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730174035999994, 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:44.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730164618999993, 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.216 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730143428000005, 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.220 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:44.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.224 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730126319999997, 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.228 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730109377000005, 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.232 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730092998000003, 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:44.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730051638000006, 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.237 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:44.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.730031956999994, 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.241 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:44.245 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 51.325143509999997, 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:44.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:44.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:44.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:44.251 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:44.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:44.252 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:44.252+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:22:44.352 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:44.352+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:44.410 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:44.410 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:44.410 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:44.411 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:44.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:44.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:44.461 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:44.461+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:44.553 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:44.554+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:44.559 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:44.559 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:44.562 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:44.562 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:44.562 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:44.563 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:44.563 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:44.571 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:44.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:44.590 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:44.606 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:44.622 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:44.638 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:44.638 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:44.654 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:44.654+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:44.654 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:44.654 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:44.670 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:44.670 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:44.713 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:44.714 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:44.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:44.766 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:44.766+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:44.825 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:44.825 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:44.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:44.856 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:44.856+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:44.957 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:44.957+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:44.977 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:44.977 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:44.985 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:45.058 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:45.058+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:45.087 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.087 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:45.159 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:45.159+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:45.198 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.199 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:45.259 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:45.260+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:45.316 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.316 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:45.360 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:45.361+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:45.462 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:45.462+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:45.470 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.470 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:45.563 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:45.563+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:45.621 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.622 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:45.663 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:45.664+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:45.764 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:45.765+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:45.772 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.772 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:45.866 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:45.866+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:45.883 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.883 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:45.891 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:45.967 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:45.967+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:45.994 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:45.994 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.002 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:46.087 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:46.087+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:46.146 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.146 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:46.168 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:46.169+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:46.269 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:46.269+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:46.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:22:46.303 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.303 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:46.371 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:46.371+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:46.426 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.426 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.435 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:46.479 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:46.480+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:46.537 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.538 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.546 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:46.572 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:46.572+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:46.672 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:46.672+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:46.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:46.689 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.689 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:46.781 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:46.782+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:46.800 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:46.800 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:46.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:46.816 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:46.831 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:46.832 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:46.840 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:46.840 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:46.847 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:46.847 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:46.848 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:46.863 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:46.864 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:46.874 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:46.874+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:46.880 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:46.880 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:46.896 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:46.896 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:46.912 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:46.974 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:46.974+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:46.991 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:46.991 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:46.991 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:46.991 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:46.991 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:46.999 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:47.075 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:47.075+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:47.103 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.103 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:47.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:47.103 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:47.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.101139276000001, 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:22:47.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:22:47.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:47.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:47.109 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.604519456, 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:22:47.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:22:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.604493886, 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.603539498, 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:22:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:22:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.603224997, 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.601809743, 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:22:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.600762293000001, 2024-09-21T14:22:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.599478878999999, 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:22:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.598157652999999, 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:22:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.597116719000001, 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:22:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:22:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.596865278999999, 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.59581953, 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.595432588, 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.594497004000001, 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.594404335, 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.590323117000001, 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.590140285, 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:22:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:22:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.589816573, 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:22:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:22:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.58969512, 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:22:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.588436521, 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:22:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.194 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:47.196 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:47.196 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:47.196+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:47.279 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:22:47.333 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:47.333+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:47.350 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.350 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:47.350 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:47.350 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:47.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:47.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:47.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:47.366 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:22:47.366 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:22:47.410 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:47.410+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:47.467 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.467 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:47.467 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:47.467 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:47.467 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:47.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:47.498 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:47.498+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:47.579 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8101 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.147051606000005, 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014039795000000001, 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8101 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431804+0000", 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431806+0000", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3879999999999998e-06 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431812+0000", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5709999999999997e-06 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431816+0000", 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7990000000000001e-06 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445682+0000", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013865219999999999 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445763+0000", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1641e-05 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445844+0000", 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0618999999999995e-05 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.147034963999999, 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:22:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.147009913999995, 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:22:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:22:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146991761999999, 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:22:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:22:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:22:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146968525000005, 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:22:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:22:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146947458, 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146931097999996, 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:22:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146900818000006, 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146885679999997, 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:22:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146866083999996, 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:22:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:22:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:22:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146857307000005, 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:22:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:22:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146844946000002, 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:22:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146835529000001, 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.634 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146814337999999, 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:22:47.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:22:47.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146797230000004, 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:22:47.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:22:47.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146780286999999, 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:22:47.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146763907999997, 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:47.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146722548, 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.652 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:22:47.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:22:47.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.146702867000002, 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:22:47.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 54.741814419999997, 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:47.660 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:22:47.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.665 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:47.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:47.666 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:47.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:47.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:47.667+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 100, 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:22:47.265303+0000", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:22:31.158403+0000", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5804032, 2024-09-21T14:22:47.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:22:47.670 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:47.671 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:47.672 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:22:47.673 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:47.674 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:47.675 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.9800000190734863, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.9800000190734863, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-21T14:22:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:47.677 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:47.678 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:47.679 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:47.680 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:47.681 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:22:47.682 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:22:47.683 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:22:47.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "100", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 55, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 100, 2024-09-21T14:22:47.685 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:22:47.686 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:22:47.687 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.2400000095367432, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.2400000095367432, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:22:47.688 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.689 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 94, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.691 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.692 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:22:47.693 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 82, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:22:47.699 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.700 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:22:47.701 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:22:47.702 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.703 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.704 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.706 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:22:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.715 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.717 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.718 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:22:47.721 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.722 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.727 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:22:47.728 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.729 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:22:47.732 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:22:47.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.734 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:47.735 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:22:47.736 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.737 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "1.1", 2024-09-21T14:22:47.739 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.d", 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: 2 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.e", 2024-09-21T14:22:47.740 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: 7 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [ 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.13", 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "mappings": [ 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "from": 4, 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: "to": 5 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.741 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:22:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [ 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "snaps": [ 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "begin": 46, 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "length": 5 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [ 2024-09-21T14:22:47.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "snaps": [ 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "begin": 46, 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "length": 1 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:22:47.744 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:22:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:22:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:22:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:22:47.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:47.745 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:47.746 INFO:tasks.thrashosds.thrasher:j is {'epoch': 100, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:22:47.265303+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:22:31.158403+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.9800000190734863, 'score_stable': 2.9800000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.1', 'osds': [5, 4]}, {'pgid': '3.d', 'osds': [6, 2]}, {'pgid': '3.e', 'osds': [4, 7]}], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 4, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 46, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 46, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:22:47.746 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:22:47.790 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:47.790+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:47.847 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.847 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:47.847 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:47.847 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:47.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:47.869 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:47.869+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:47.970 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:47.970+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:47.997 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:47.998 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:48.006 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:48.019 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:22:48.020 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:22:48.032 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-21T14:22:47.277432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1794731851,"num_objects":2186,"num_object_clones":624,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":852,"num_read":134899,"num_read_kb":942797,"num_write":19683,"num_write_kb":505570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":265291546,"num_keys_recovered":619,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15797,"ondisk_log_size":15797,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":307112,"kb_used_data":80944,"kb_used_omap":7,"kb_used_meta":226040,"kb_avail":471552088,"statfs":{"total":483183820800,"available":482869338112,"internally_reserved":0,"allocated":82886656,"data_stored":784500233,"data_compressed":46195186,"data_compressed_allocated":54251520,"data_compressed_original":762856429,"omap_allocated":7964,"internal_metadata":231465188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028005"},"pg_stats":[{"pgid":"3.1f","version":"95'310","reported_seq":2993,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.401291+0000","last_change":"2024-09-21T14:22:35.217665+0000","last_active":"2024-09-21T14:22:42.401291+0000","last_peered":"2024-09-21T14:22:42.401291+0000","last_clean":"2024-09-21T14:22:42.401291+0000","last_became_active":"2024-09-21T14:22:35.216886+0000","last_became_peered":"2024-09-21T14:22:35.216886+0000","last_unstale":"2024-09-21T14:22:42.401291+0000","last_undegraded":"2024-09-21T14:22:42.401291+0000","last_fullsized":"2024-09-21T14:22:42.401291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:49.746608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028564699999999998,"stat_sum":{"num_bytes":25000686,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":23,"num_read":3451,"num_read_kb":17450,"num_write":311,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3807877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"95'276","reported_seq":1861,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.318611+0000","last_change":"2024-09-21T14:22:29.159013+0000","last_active":"2024-09-21T14:22:42.318611+0000","last_peered":"2024-09-21T14:22:42.318611+0000","last_clean":"2024-09-21T14:22:42.318611+0000","last_became_active":"2024-09-21T14:22:29.158180+0000","last_became_peered":"2024-09-21T14:22:29.158180+0000","last_unstale":"2024-09-21T14:22:42.318611+0000","last_undegraded":"2024-09-21T14:22:42.318611+0000","last_fullsized":"2024-09-21T14:22:42.318611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:47.183599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032714400000000002,"stat_sum":{"num_bytes":62439571,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":1803,"num_read_kb":53144,"num_write":292,"num_write_kb":35425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":32363151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"95'373","reported_seq":3099,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.003123+0000","last_change":"2024-09-21T14:22:36.211217+0000","last_active":"2024-09-21T14:22:42.003123+0000","last_peered":"2024-09-21T14:22:42.003123+0000","last_clean":"2024-09-21T14:22:42.003123+0000","last_became_active":"2024-09-21T14:22:36.210300+0000","last_became_peered":"2024-09-21T14:22:36.210300+0000","last_unstale":"2024-09-21T14:22:42.003123+0000","last_undegraded":"2024-09-21T14:22:42.003123+0000","last_fullsized":"2024-09-21T14:22:42.003123+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:16.513441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089526899999999997,"stat_sum":{"num_bytes":26817953,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":26,"num_read":3720,"num_read_kb":19908,"num_write":402,"num_write_kb":10357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14805334,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"95'357","reported_seq":5522,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.439178+0000","last_change":"2024-09-21T14:22:35.214957+0000","last_active":"2024-09-21T14:22:42.439178+0000","last_peered":"2024-09-21T14:22:42.439178+0000","last_clean":"2024-09-21T14:22:42.439178+0000","last_became_active":"2024-09-21T14:22:35.214138+0000","last_became_peered":"2024-09-21T14:22:35.214138+0000","last_unstale":"2024-09-21T14:22:42.439178+0000","last_undegraded":"2024-09-21T14:22:42.439178+0000","last_fullsized":"2024-09-21T14:22:42.439178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.144123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059858800000000005,"stat_sum":{"num_bytes":82734989,"num_objects":83,"num_object_clones":24,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":24,"num_read":5800,"num_read_kb":82480,"num_write":402,"num_write_kb":30694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":26488863,"num_keys_recovered":104,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"95'431","reported_seq":4286,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.401012+0000","last_change":"2024-09-21T14:21:20.146155+0000","last_active":"2024-09-21T14:22:42.401012+0000","last_peered":"2024-09-21T14:22:42.401012+0000","last_clean":"2024-09-21T14:22:42.401012+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:22:42.401012+0000","last_undegraded":"2024-09-21T14:22:42.401012+0000","last_fullsized":"2024-09-21T14:22:42.401012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481958,"stat_sum":{"num_bytes":91139116,"num_objects":76,"num_object_clones":26,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":29,"num_read":5618,"num_read_kb":42191,"num_write":394,"num_write_kb":19664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"95'440","reported_seq":4781,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.435960+0000","last_change":"2024-09-21T14:22:35.216312+0000","last_active":"2024-09-21T14:22:42.435960+0000","last_peered":"2024-09-21T14:22:42.435960+0000","last_clean":"2024-09-21T14:22:42.435960+0000","last_became_active":"2024-09-21T14:22:35.215561+0000","last_became_peered":"2024-09-21T14:22:35.215561+0000","last_unstale":"2024-09-21T14:22:42.435960+0000","last_undegraded":"2024-09-21T14:22:42.435960+0000","last_fullsized":"2024-09-21T14:22:42.435960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:05.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054410299999999999,"stat_sum":{"num_bytes":79951360,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":27,"num_read":6027,"num_read_kb":65152,"num_write":509,"num_write_kb":34379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":29327918,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"95'715","reported_seq":5507,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.337591+0000","last_change":"2024-09-21T14:22:35.218275+0000","last_active":"2024-09-21T14:22:42.337591+0000","last_peered":"2024-09-21T14:22:42.337591+0000","last_clean":"2024-09-21T14:22:42.337591+0000","last_became_active":"2024-09-21T14:22:35.217004+0000","last_became_peered":"2024-09-21T14:22:35.217004+0000","last_unstale":"2024-09-21T14:22:42.337591+0000","last_undegraded":"2024-09-21T14:22:42.337591+0000","last_fullsized":"2024-09-21T14:22:42.337591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:44.918657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503091,"stat_sum":{"num_bytes":63907389,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":7244,"num_read_kb":30114,"num_write":999,"num_write_kb":11009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":7435469,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"95'242","reported_seq":1802,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.000192+0000","last_change":"2024-09-21T14:21:20.150030+0000","last_active":"2024-09-21T14:22:42.000192+0000","last_peered":"2024-09-21T14:22:42.000192+0000","last_clean":"2024-09-21T14:22:42.000192+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:22:42.000192+0000","last_undegraded":"2024-09-21T14:22:42.000192+0000","last_fullsized":"2024-09-21T14:22:42.000192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039522199999999998,"stat_sum":{"num_bytes":32252883,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":1637,"num_read_kb":12528,"num_write":227,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"95'672","reported_seq":2869,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.426624+0000","last_change":"2024-09-21T14:22:35.217924+0000","last_active":"2024-09-21T14:22:42.426624+0000","last_peered":"2024-09-21T14:22:42.426624+0000","last_clean":"2024-09-21T14:22:42.426624+0000","last_became_active":"2024-09-21T14:22:35.217012+0000","last_became_peered":"2024-09-21T14:22:35.217012+0000","last_unstale":"2024-09-21T14:22:42.426624+0000","last_undegraded":"2024-09-21T14:22:42.426624+0000","last_fullsized":"2024-09-21T14:22:42.426624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:02.683415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046787400000000001,"stat_sum":{"num_bytes":20745235,"num_objects":74,"num_object_clones":23,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":31,"num_read":4519,"num_read_kb":10023,"num_write":894,"num_write_kb":13973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":5304982,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"95'443","reported_seq":2505,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.255038+0000","last_change":"2024-09-21T14:22:29.162378+0000","last_active":"2024-09-21T14:22:42.255038+0000","last_peered":"2024-09-21T14:22:42.255038+0000","last_clean":"2024-09-21T14:22:42.255038+0000","last_became_active":"2024-09-21T14:22:29.161682+0000","last_became_peered":"2024-09-21T14:22:29.161682+0000","last_unstale":"2024-09-21T14:22:42.255038+0000","last_undegraded":"2024-09-21T14:22:42.255038+0000","last_fullsized":"2024-09-21T14:22:42.255038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:15.765317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022715499999999999,"stat_sum":{"num_bytes":24812317,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":3175,"num_read_kb":13842,"num_write":601,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":3589748,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"95'726","reported_seq":2620,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.415673+0000","last_change":"2024-09-21T14:21:20.151980+0000","last_active":"2024-09-21T14:22:42.415673+0000","last_peered":"2024-09-21T14:22:42.415673+0000","last_clean":"2024-09-21T14:22:42.415673+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:22:42.415673+0000","last_undegraded":"2024-09-21T14:22:42.415673+0000","last_fullsized":"2024-09-21T14:22:42.415673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067804499999999995,"stat_sum":{"num_bytes":66471546,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":4814,"num_read_kb":24605,"num_write":1027,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"95'386","reported_seq":2917,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.446232+0000","last_change":"2024-09-21T14:22:29.165906+0000","last_active":"2024-09-21T14:22:42.446232+0000","last_peered":"2024-09-21T14:22:42.446232+0000","last_clean":"2024-09-21T14:22:42.446232+0000","last_became_active":"2024-09-21T14:22:29.164131+0000","last_became_peered":"2024-09-21T14:22:29.164131+0000","last_unstale":"2024-09-21T14:22:42.446232+0000","last_undegraded":"2024-09-21T14:22:42.446232+0000","last_fullsized":"2024-09-21T14:22:42.446232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:06.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010765359999999999,"stat_sum":{"num_bytes":42920398,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":4585,"num_read_kb":16522,"num_write":445,"num_write_kb":9899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":6514761,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244101+0000","last_change":"2024-09-21T14:22:32.174490+0000","last_active":"2024-09-21T14:22:40.244101+0000","last_peered":"2024-09-21T14:22:40.244101+0000","last_clean":"2024-09-21T14:22:40.244101+0000","last_became_active":"2024-09-21T14:22:32.174175+0000","last_became_peered":"2024-09-21T14:22:32.174175+0000","last_unstale":"2024-09-21T14:22:40.244101+0000","last_undegraded":"2024-09-21T14:22:40.244101+0000","last_fullsized":"2024-09-21T14:22:40.244101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:54.846715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"95'367","reported_seq":2756,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.430085+0000","last_change":"2024-09-21T14:21:52.138940+0000","last_active":"2024-09-21T14:22:42.430085+0000","last_peered":"2024-09-21T14:22:42.430085+0000","last_clean":"2024-09-21T14:22:42.430085+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:22:42.430085+0000","last_undegraded":"2024-09-21T14:22:42.430085+0000","last_fullsized":"2024-09-21T14:22:42.430085+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030815399999999999,"stat_sum":{"num_bytes":55941489,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":27,"num_read":3356,"num_read_kb":12735,"num_write":368,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244007+0000","last_change":"2024-09-21T14:22:26.141620+0000","last_active":"2024-09-21T14:22:40.244007+0000","last_peered":"2024-09-21T14:22:40.244007+0000","last_clean":"2024-09-21T14:22:40.244007+0000","last_became_active":"2024-09-21T14:22:26.141314+0000","last_became_peered":"2024-09-21T14:22:26.141314+0000","last_unstale":"2024-09-21T14:22:40.244007+0000","last_undegraded":"2024-09-21T14:22:40.244007+0000","last_fullsized":"2024-09-21T14:22:40.244007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:24:21.823458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"95'447","reported_seq":2898,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.336575+0000","last_change":"2024-09-21T14:21:20.756219+0000","last_active":"2024-09-21T14:22:42.336575+0000","last_peered":"2024-09-21T14:22:42.336575+0000","last_clean":"2024-09-21T14:22:42.336575+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:22:42.336575+0000","last_undegraded":"2024-09-21T14:22:42.336575+0000","last_fullsized":"2024-09-21T14:22:42.336575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048965100000000002,"stat_sum":{"num_bytes":46236746,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":3855,"num_read_kb":18020,"num_write":582,"num_write_kb":5955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244012+0000","last_change":"2024-09-21T14:22:26.141948+0000","last_active":"2024-09-21T14:22:40.244012+0000","last_peered":"2024-09-21T14:22:40.244012+0000","last_clean":"2024-09-21T14:22:40.244012+0000","last_became_active":"2024-09-21T14:22:26.141693+0000","last_became_peered":"2024-09-21T14:22:26.141693+0000","last_unstale":"2024-09-21T14:22:40.244012+0000","last_undegraded":"2024-09-21T14:22:40.244012+0000","last_fullsized":"2024-09-21T14:22:40.244012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:29:48.520733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"95'328","reported_seq":2946,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.190618+0000","last_change":"2024-09-21T14:21:20.146314+0000","last_active":"2024-09-21T14:22:42.190618+0000","last_peered":"2024-09-21T14:22:42.190618+0000","last_clean":"2024-09-21T14:22:42.190618+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:22:42.190618+0000","last_undegraded":"2024-09-21T14:22:42.190618+0000","last_fullsized":"2024-09-21T14:22:42.190618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053372700000000001,"stat_sum":{"num_bytes":49907046,"num_objects":77,"num_object_clones":24,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":31,"num_read":2797,"num_read_kb":36163,"num_write":322,"num_write_kb":12309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.247905+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:22:40.247905+0000","last_peered":"2024-09-21T14:22:40.247905+0000","last_clean":"2024-09-21T14:22:40.247905+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:22:40.247905+0000","last_undegraded":"2024-09-21T14:22:40.247905+0000","last_fullsized":"2024-09-21T14:22:40.247905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"95'435","reported_seq":5116,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.406701+0000","last_change":"2024-09-21T14:21:20.146386+0000","last_active":"2024-09-21T14:22:42.406701+0000","last_peered":"2024-09-21T14:22:42.406701+0000","last_clean":"2024-09-21T14:22:42.406701+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:22:42.406701+0000","last_undegraded":"2024-09-21T14:22:42.406701+0000","last_fullsized":"2024-09-21T14:22:42.406701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058816300000000005,"stat_sum":{"num_bytes":56555943,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":25,"num_read":6236,"num_read_kb":31013,"num_write":462,"num_write_kb":9243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"95'284","reported_seq":2450,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.401739+0000","last_change":"2024-09-21T14:21:52.139305+0000","last_active":"2024-09-21T14:22:42.401739+0000","last_peered":"2024-09-21T14:22:42.401739+0000","last_clean":"2024-09-21T14:22:42.401739+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:22:42.401739+0000","last_undegraded":"2024-09-21T14:22:42.401739+0000","last_fullsized":"2024-09-21T14:22:42.401739+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000342846,"stat_sum":{"num_bytes":41991623,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":23,"num_read":3088,"num_read_kb":21572,"num_write":266,"num_write_kb":9367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244319+0000","last_change":"2024-09-21T14:22:29.157860+0000","last_active":"2024-09-21T14:22:40.244319+0000","last_peered":"2024-09-21T14:22:40.244319+0000","last_clean":"2024-09-21T14:22:40.244319+0000","last_became_active":"2024-09-21T14:22:29.157461+0000","last_became_peered":"2024-09-21T14:22:29.157461+0000","last_unstale":"2024-09-21T14:22:40.244319+0000","last_undegraded":"2024-09-21T14:22:40.244319+0000","last_fullsized":"2024-09-21T14:22:40.244319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:27.251019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":107,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.239061+0000","last_change":"2024-09-21T14:22:32.181750+0000","last_active":"2024-09-21T14:22:40.239061+0000","last_peered":"2024-09-21T14:22:40.239061+0000","last_clean":"2024-09-21T14:22:40.239061+0000","last_became_active":"2024-09-21T14:22:32.177114+0000","last_became_peered":"2024-09-21T14:22:32.177114+0000","last_unstale":"2024-09-21T14:22:40.239061+0000","last_undegraded":"2024-09-21T14:22:40.239061+0000","last_fullsized":"2024-09-21T14:22:40.239061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:00.138436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"95'695","reported_seq":3043,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.419373+0000","last_change":"2024-09-21T14:21:20.149907+0000","last_active":"2024-09-21T14:22:42.419373+0000","last_peered":"2024-09-21T14:22:42.419373+0000","last_clean":"2024-09-21T14:22:42.419373+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:22:42.419373+0000","last_undegraded":"2024-09-21T14:22:42.419373+0000","last_fullsized":"2024-09-21T14:22:42.419373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045277300000000001,"stat_sum":{"num_bytes":87892680,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":35,"num_read":5097,"num_read_kb":124477,"num_write":945,"num_write_kb":60245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":96,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.241160+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:22:40.241160+0000","last_peered":"2024-09-21T14:22:40.241160+0000","last_clean":"2024-09-21T14:22:40.241160+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:22:40.241160+0000","last_undegraded":"2024-09-21T14:22:40.241160+0000","last_fullsized":"2024-09-21T14:22:40.241160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"95'946","reported_seq":2129,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.424047+0000","last_change":"2024-09-21T14:21:52.139498+0000","last_active":"2024-09-21T14:22:42.424047+0000","last_peered":"2024-09-21T14:22:42.424047+0000","last_clean":"2024-09-21T14:22:42.424047+0000","last_became_active":"2024-09-21T14:21:52.138744+0000","last_became_peered":"2024-09-21T14:21:52.138744+0000","last_unstale":"2024-09-21T14:22:42.424047+0000","last_undegraded":"2024-09-21T14:22:42.424047+0000","last_fullsized":"2024-09-21T14:22:42.424047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:16:08.802429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085972700000000002,"stat_sum":{"num_bytes":33352125,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":23,"num_read":4410,"num_read_kb":16150,"num_write":1427,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2088730,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244385+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:22:40.244385+0000","last_peered":"2024-09-21T14:22:40.244385+0000","last_clean":"2024-09-21T14:22:40.244385+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:22:40.244385+0000","last_undegraded":"2024-09-21T14:22:40.244385+0000","last_fullsized":"2024-09-21T14:22:40.244385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'500","reported_seq":2340,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.442742+0000","last_change":"2024-09-21T14:21:52.139320+0000","last_active":"2024-09-21T14:22:42.442742+0000","last_peered":"2024-09-21T14:22:42.442742+0000","last_clean":"2024-09-21T14:22:42.442742+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:22:42.442742+0000","last_undegraded":"2024-09-21T14:22:42.442742+0000","last_fullsized":"2024-09-21T14:22:42.442742+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000309776,"stat_sum":{"num_bytes":82584805,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":26,"num_read":3343,"num_read_kb":12189,"num_write":632,"num_write_kb":5621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T14:22:40.244273+0000","last_change":"2024-09-21T14:22:32.172034+0000","last_active":"2024-09-21T14:22:40.244273+0000","last_peered":"2024-09-21T14:22:40.244273+0000","last_clean":"2024-09-21T14:22:40.244273+0000","last_became_active":"2024-09-21T14:22:32.171774+0000","last_became_peered":"2024-09-21T14:22:32.171774+0000","last_unstale":"2024-09-21T14:22:40.244273+0000","last_undegraded":"2024-09-21T14:22:40.244273+0000","last_fullsized":"2024-09-21T14:22:40.244273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:41.624554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"95'560","reported_seq":3221,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.401704+0000","last_change":"2024-09-21T14:21:20.145946+0000","last_active":"2024-09-21T14:22:42.401704+0000","last_peered":"2024-09-21T14:22:42.401704+0000","last_clean":"2024-09-21T14:22:42.401704+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:22:42.401704+0000","last_undegraded":"2024-09-21T14:22:42.401704+0000","last_fullsized":"2024-09-21T14:22:42.401704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033099699999999998,"stat_sum":{"num_bytes":81647925,"num_objects":71,"num_object_clones":24,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":23,"num_read":5405,"num_read_kb":15633,"num_write":657,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"95'546","reported_seq":4008,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.148010+0000","last_change":"2024-09-21T14:22:29.167445+0000","last_active":"2024-09-21T14:22:42.148010+0000","last_peered":"2024-09-21T14:22:42.148010+0000","last_clean":"2024-09-21T14:22:42.148010+0000","last_became_active":"2024-09-21T14:22:29.166889+0000","last_became_peered":"2024-09-21T14:22:29.166889+0000","last_unstale":"2024-09-21T14:22:42.148010+0000","last_undegraded":"2024-09-21T14:22:42.148010+0000","last_fullsized":"2024-09-21T14:22:42.148010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:48.958859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011638239999999999,"stat_sum":{"num_bytes":134862950,"num_objects":68,"num_object_clones":17,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":23,"num_read":6741,"num_read_kb":83842,"num_write":614,"num_write_kb":55663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":51428954,"num_keys_recovered":47,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'360","reported_seq":1663,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.255561+0000","last_change":"2024-09-21T14:22:35.218723+0000","last_active":"2024-09-21T14:22:42.255561+0000","last_peered":"2024-09-21T14:22:42.255561+0000","last_clean":"2024-09-21T14:22:42.255561+0000","last_became_active":"2024-09-21T14:22:35.217329+0000","last_became_peered":"2024-09-21T14:22:35.217329+0000","last_unstale":"2024-09-21T14:22:42.255561+0000","last_undegraded":"2024-09-21T14:22:42.255561+0000","last_fullsized":"2024-09-21T14:22:42.255561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:17:58.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001127255,"stat_sum":{"num_bytes":25045900,"num_objects":70,"num_object_clones":23,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":1213,"num_read_kb":8516,"num_write":368,"num_write_kb":7224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":11062623,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"95'387","reported_seq":2122,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.255345+0000","last_change":"2024-09-21T14:21:20.756411+0000","last_active":"2024-09-21T14:22:42.255345+0000","last_peered":"2024-09-21T14:22:42.255345+0000","last_clean":"2024-09-21T14:22:42.255345+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:22:42.255345+0000","last_undegraded":"2024-09-21T14:22:42.255345+0000","last_fullsized":"2024-09-21T14:22:42.255345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031795700000000001,"stat_sum":{"num_bytes":50993723,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":28,"num_read":2515,"num_read_kb":10244,"num_write":520,"num_write_kb":7163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"95'628","reported_seq":2093,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.247782+0000","last_change":"2024-09-21T14:21:20.152184+0000","last_active":"2024-09-21T14:22:42.247782+0000","last_peered":"2024-09-21T14:22:42.247782+0000","last_clean":"2024-09-21T14:22:42.247782+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:22:42.247782+0000","last_undegraded":"2024-09-21T14:22:42.247782+0000","last_fullsized":"2024-09-21T14:22:42.247782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075785800000000003,"stat_sum":{"num_bytes":48421455,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":25,"num_read":3609,"num_read_kb":17806,"num_write":838,"num_write_kb":10995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"95'476","reported_seq":2594,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.135400+0000","last_change":"2024-09-21T14:22:36.215237+0000","last_active":"2024-09-21T14:22:42.135400+0000","last_peered":"2024-09-21T14:22:42.135400+0000","last_clean":"2024-09-21T14:22:42.135400+0000","last_became_active":"2024-09-21T14:22:36.212777+0000","last_became_peered":"2024-09-21T14:22:36.212777+0000","last_unstale":"2024-09-21T14:22:42.135400+0000","last_undegraded":"2024-09-21T14:22:42.135400+0000","last_fullsized":"2024-09-21T14:22:42.135400+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:54:27.444035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010606630000000001,"stat_sum":{"num_bytes":45122762,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":3339,"num_read_kb":14288,"num_write":649,"num_write_kb":16260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5387372,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"95'918","reported_seq":3532,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.406665+0000","last_change":"2024-09-21T14:22:23.540379+0000","last_active":"2024-09-21T14:22:42.406665+0000","last_peered":"2024-09-21T14:22:42.406665+0000","last_clean":"2024-09-21T14:22:42.406665+0000","last_became_active":"2024-09-21T14:22:23.539499+0000","last_became_peered":"2024-09-21T14:22:23.539499+0000","last_unstale":"2024-09-21T14:22:42.406665+0000","last_undegraded":"2024-09-21T14:22:42.406665+0000","last_fullsized":"2024-09-21T14:22:42.406665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:00:47.787223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029404799999999999,"stat_sum":{"num_bytes":69988995,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":26,"num_read":6044,"num_read_kb":20603,"num_write":1389,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":7784436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"95'402","reported_seq":4541,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.335408+0000","last_change":"2024-09-21T14:22:36.217691+0000","last_active":"2024-09-21T14:22:42.335408+0000","last_peered":"2024-09-21T14:22:42.335408+0000","last_clean":"2024-09-21T14:22:42.335408+0000","last_became_active":"2024-09-21T14:22:36.216864+0000","last_became_peered":"2024-09-21T14:22:36.216864+0000","last_unstale":"2024-09-21T14:22:42.335408+0000","last_undegraded":"2024-09-21T14:22:42.335408+0000","last_fullsized":"2024-09-21T14:22:42.335408+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:43.577416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059732299999999997,"stat_sum":{"num_bytes":87257262,"num_objects":78,"num_object_clones":21,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":27,"num_read":5346,"num_read_kb":25467,"num_write":410,"num_write_kb":21456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6327001,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"95'346","reported_seq":2849,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.175957+0000","last_change":"2024-09-21T14:22:35.209373+0000","last_active":"2024-09-21T14:22:42.175957+0000","last_peered":"2024-09-21T14:22:42.175957+0000","last_clean":"2024-09-21T14:22:42.175957+0000","last_became_active":"2024-09-21T14:22:35.208525+0000","last_became_peered":"2024-09-21T14:22:35.208525+0000","last_unstale":"2024-09-21T14:22:42.175957+0000","last_undegraded":"2024-09-21T14:22:42.175957+0000","last_fullsized":"2024-09-21T14:22:42.175957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.874702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000475957,"stat_sum":{"num_bytes":47608795,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":31,"num_read":2767,"num_read_kb":16164,"num_write":335,"num_write_kb":7253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":5015729,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'308","reported_seq":2405,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.171108+0000","last_change":"2024-09-21T14:22:35.210233+0000","last_active":"2024-09-21T14:22:42.171108+0000","last_peered":"2024-09-21T14:22:42.171108+0000","last_clean":"2024-09-21T14:22:42.171108+0000","last_became_active":"2024-09-21T14:22:35.208396+0000","last_became_peered":"2024-09-21T14:22:35.208396+0000","last_unstale":"2024-09-21T14:22:42.171108+0000","last_undegraded":"2024-09-21T14:22:42.171108+0000","last_fullsized":"2024-09-21T14:22:42.171108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:35.143804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038340899999999997,"stat_sum":{"num_bytes":61243577,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":1890,"num_read_kb":23055,"num_write":379,"num_write_kb":18874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":22976540,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"95'427","reported_seq":2920,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.436356+0000","last_change":"2024-09-21T14:22:29.158071+0000","last_active":"2024-09-21T14:22:42.436356+0000","last_peered":"2024-09-21T14:22:42.436356+0000","last_clean":"2024-09-21T14:22:42.436356+0000","last_became_active":"2024-09-21T14:22:29.157244+0000","last_became_peered":"2024-09-21T14:22:29.157244+0000","last_unstale":"2024-09-21T14:22:42.436356+0000","last_undegraded":"2024-09-21T14:22:42.436356+0000","last_fullsized":"2024-09-21T14:22:42.436356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:24.971948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491709,"stat_sum":{"num_bytes":36945465,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4287,"num_read_kb":22473,"num_write":492,"num_write_kb":10758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5431126,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"95'946","reported_seq":4156,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T14:22:42.400710+0000","last_change":"2024-09-21T14:22:41.250813+0000","last_active":"2024-09-21T14:22:42.400710+0000","last_peered":"2024-09-21T14:22:42.400710+0000","last_clean":"2024-09-21T14:22:42.400710+0000","last_became_active":"2024-09-21T14:22:41.248710+0000","last_became_peered":"2024-09-21T14:22:41.248710+0000","last_unstale":"2024-09-21T14:22:42.400710+0000","last_undegraded":"2024-09-21T14:22:42.400710+0000","last_fullsized":"2024-09-21T14:22:42.400710+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:41.665719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074146,"stat_sum":{"num_bytes":31346755,"num_objects":76,"num_object_clones":24,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":29,"num_read":7052,"num_read_kb":24328,"num_write":1364,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":13233521,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1794141464,"num_objects":2182,"num_object_clones":624,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":852,"num_read":134783,"num_read_kb":942697,"num_write":19522,"num_write_kb":503152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":264701178,"num_keys_recovered":619,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103432192,"data_stored":1045501950,"data_compressed":61204857,"data_compressed_allocated":72192000,"data_compressed_original":1019492838,"omap_allocated":0,"internal_metadata":0},"log_size":15677,"ondisk_log_size":15677,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98865,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41684,"kb_used_data":14596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330156,"statfs":{"total":96636764160,"available":96594079744,"internally_reserved":0,"allocated":14946304,"data_stored":132788449,"data_compressed":7875785,"data_compressed_allocated":9416704,"data_compressed_original":128767510,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.435},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.378}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39100000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.32400000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.312}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.45200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54748,"kb_used_data":11348,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94317092,"statfs":{"total":96636764160,"available":96580702208,"internally_reserved":0,"allocated":11620352,"data_stored":102582186,"data_compressed":6200929,"data_compressed_allocated":7057408,"data_compressed_original":99172642,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.625}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36199999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50244,"kb_used_data":19124,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94321596,"statfs":{"total":96636764160,"available":96585314304,"internally_reserved":0,"allocated":19582976,"data_stored":197245603,"data_compressed":11363615,"data_compressed_allocated":13901824,"data_compressed_original":192866696,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.248},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.26700000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.311}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.7430000000000003,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.67800000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.48699999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.498}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":19324,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":19787776,"data_stored":183952233,"data_compressed":11077675,"data_compressed_allocated":12750848,"data_compressed_original":178555649,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.503},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.312},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.46200000000000002}]}]},{"osd":3,"up_from":0,"seq":55834574870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":0,"seq":55834574870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87856,"kb_used_data":16552,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94283984,"statfs":{"total":96636764160,"available":96546799616,"internally_reserved":0,"allocated":16949248,"data_stored":167931762,"data_compressed":9677182,"data_compressed_allocated":11124736,"data_compressed_original":163493932,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.35299999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.33900000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.40300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.39200000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.438},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.42399999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15613952,"data_stored":165019526,"data_compressed":9564614,"data_compressed_allocated":10940416,"data_compressed_original":161126444,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12337152,"data_stored":119058163,"data_compressed":6888762,"data_compressed_allocated":8253440,"data_compressed_original":115661228,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":140569390,"data_compressed":7948729,"data_compressed_allocated":9515008,"data_compressed_original":137234462,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18022400,"data_stored":178275923,"data_compressed":10764915,"data_compressed_allocated":12296192,"data_compressed_original":173566721,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18337792,"data_stored":195842200,"data_compressed":11312724,"data_compressed_allocated":13832192,"data_compressed_original":192038802,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10362880,"data_stored":101122964,"data_compressed":6142179,"data_compressed_allocated":6975488,"data_compressed_original":98279714,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13496320,"data_stored":129539069,"data_compressed":7694768,"data_compressed_allocated":9166848,"data_compressed_original":126104608,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:22:48.034 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [6, 5] 2024-09-21T14:22:48.035 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.3', '6', '5'] 2024-09-21T14:22:48.035 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.3 6 5 2024-09-21T14:22:48.090 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:48.090+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:48.150 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553808469000003, 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.151 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553790954999997, 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.155 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:48.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.159 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553773700999997, 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.160 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:48.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:48.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553745851000002, 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.164 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:48.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:48.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:48.169 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553724247000005, 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:48.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553707248999999, 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:48.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:48.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:48.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553690228999997, 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:48.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553672778999996, 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.187 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553655641000006, 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:48.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:48.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.191 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553632414000006, 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:48.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:48.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:48.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553616140000003, 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.200 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553591768000004, 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:48.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553569201000002, 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.205 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:48.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.209 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553552280000005, 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553535780999994, 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:48.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553514784000001, 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.219 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:48.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 78.553492726000002, 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.223 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:48.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:48.227 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.379421854, 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:48.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.23718873, 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.233 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.004478654, 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:22:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:48.242 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:48.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:48.244 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:48.243+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:48.344 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:48.344+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:48.358 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:48.358 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:48.358 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:48.358 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:48.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:48.452 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:48.452+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:48.510 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:48.511 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:48.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:48.546 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:48.546+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:48.647 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:48.647+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:48.663 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.510739692000001, 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:22:48.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.167855986999999, 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.670 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.167840228999999, 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:22:48.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:22:48.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:22:48.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.167818889999999, 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:48.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:22:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.683 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.167803082000001, 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:22:48.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.167786808000001, 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:22:48.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.16776808, 2024-09-21T14:22:48.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:22:48.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:22:48.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166798641, 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:22:48.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:22:48.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166776008999999, 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166759369999999, 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:22:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:48.710 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166743393999999, 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:48.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:22:48.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166726242999999, 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.715 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166709481, 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166502057000001, 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:48.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:22:48.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.166156849, 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:22:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.732 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.320285285000001, 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:48.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:22:48.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.309745762, 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:22:48.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:22:48.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:22:48.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:22:48.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:22:48.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.304531385000001, 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:22:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:48.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.2838198, 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:22:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:22:48.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:22:48.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.280746647999999, 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:48.762 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:48.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:22:48.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:22:48.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:22:48.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:48.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:22:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:48.770 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:48.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:48.772 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:48.771+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:48.872 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:48.872+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:48.874 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:48.874 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:48.874 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:48.874 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:48.883 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:48.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:48.973 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:48.973+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:49.030 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:49.031 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:49.031 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:49.031 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:49.031 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:49.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:49.045 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:49.045 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:49.074 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:49.075+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:49.110 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:49.110 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:49.111 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:49.111 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:49.113 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:49.130 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:49.130 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:49.147 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:49.147 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:49.151 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.151 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:49.165 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:49.165 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:49.176 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:49.176+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:49.267 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.267 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:49.277 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:49.277+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:set 3.3 pg_upmap mapping to [6,5] 2024-09-21T14:22:49.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:22:49.378 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:49.378+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:49.388 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.388 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:49.481 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:49.481+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:49.541 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.542 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.550 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:49.579 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:49.579+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:49.679 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:49.679+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:49.692 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.692 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:49.780 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:49.781+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:49.843 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.843 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:49.882 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:49.882+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:49.952 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:49.953 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:49.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:49.983 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:49.983+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:50.076 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.077 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.084 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:50.084+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:50.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:50.184 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:50.184+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:50.228 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.228 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.237 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:50.287 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:22:50.320 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:50.321+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:50.381 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.381 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.385 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:50.386+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:50.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:50.486 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:50.486+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:50.533 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.533 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:50.587 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:50.587+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:50.645 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.645 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:50.698 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:50.698+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.765 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:50.806 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:50.806+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:50.890 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:50.890+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:50.908 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:50.908 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:50.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:50.991 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:50.991+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:51.059 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:51.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:51.091 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:51.091+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:51.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:51.185 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:51.185 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:51.196 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:51.221 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:51.221+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:51.293 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:51.294+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:51.319 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:51.319 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:51.323 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:51.323 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:51.329 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:51.343 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:51.343 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:51.359 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:51.359 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:51.375 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:51.375 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:51.392 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:51.392 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:51.394 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:51.394+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:51.409 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:51.409 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:51.425 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:51.426 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:51.443 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:51.443 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:51.484 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:51.484 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:51.484 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:51.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:51.495 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:51.495+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.596 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999107889000001, 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:51.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:22:51.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999090374999994, 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:51.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999073120999995, 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.606 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:22:51.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:22:51.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:22:51.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999045271, 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:22:51.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:22:51.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999023667000003, 2024-09-21T14:22:51.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:22:51.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.999006668999996, 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:22:51.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:22:51.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998989648999995, 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.624 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:22:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:22:51.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998972198999994, 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.628 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:22:51.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:51.632 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998955061000004, 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:22:51.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.636 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998931834000004, 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.637 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:22:51.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.99891556, 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998891188000002, 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:51.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.649 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998868621, 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:22:51.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:22:51.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998851700000003, 2024-09-21T14:22:51.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:22:51.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:22:51.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998835201000006, 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:22:51.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998814203999999, 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:22:51.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 81.998792146, 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:22:51.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:22:51.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824721273999998, 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:22:51.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:22:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:22:51.677 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.682488150000001, 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:51.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 26.449778074000001, 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:22:51.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:22:51.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:51.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:51.687 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:51.688 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:51.596+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:51.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:51.730 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:51.730+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:51.791 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:51.791 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:51.791 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:51.791 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:51.796 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:51.797+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:51.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:51.897 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:51.897+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:51.901 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:51.902 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:51.902 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:51.902 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:51.902 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:51.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:51.998 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:51.999+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 23.902033457000002, 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:22:52.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:22:52.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559149752, 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:22:52.060 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:22:52.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:22:52.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559133994, 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:22:52.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:22:52.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559112655, 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.069 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:52.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:22:52.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559096846999999, 2024-09-21T14:22:52.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.077 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559080572999999, 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.081 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.559061844999999, 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.082 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:22:52.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:22:52.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558092406, 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.086 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558069774, 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:22:52.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558053135000002, 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:22:52.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:22:52.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558037159000001, 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:22:52.099 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:22:52.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:22:52.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558020008, 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:22:52.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:22:52.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.558003245999998, 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.108 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:22:52.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.557795821999999, 2024-09-21T14:22:52.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:22:52.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.557450614, 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.711579049999999, 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.121 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:22:52.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:22:52.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:22:52.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:22:52.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.701039527000001, 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:22:52.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:22:52.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:22:52.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:22:52.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.695825149999999, 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:22:52.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.675113565, 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:22:52.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:22:52.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:22:52.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.672040413, 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.149 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:22:52.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:22:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:22:52.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.156 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:52.158 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:52.158+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:52.258 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:52.258+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:52.263 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.263 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:52.263 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:52.263 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:52.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:22:52.359 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:52.359+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:52.418 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.418 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:52.418 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:52.419 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:52.419 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:52.469 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:52.469+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:52.561 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:52.561+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:52.573 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 79.352072168999996, 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:52.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:52.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860247706999999, 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:52.579 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:52.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860230002999998, 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:52.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860214331999998, 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:52.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:52.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860198818000001, 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.593 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860183751000001, 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:52.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.601 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860168555999998, 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:52.605 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860153449999999, 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.606 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:52.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860138339999999, 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:52.610 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:52.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860122975000003, 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.618 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860107515999999, 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:52.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:52.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860092483000003, 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:52.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:52.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860076593999999, 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.631 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860050917999999, 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:52.635 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860035633000003, 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.636 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:52.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.860013668999997, 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.640 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:52.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.644 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 54.859997686, 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:52.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:52.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:52.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.809456298000001, 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:22:52.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:52.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.345402541999999, 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:22:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:22:52.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:22:52.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 24.412491557999999, 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:22:52.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:22:52.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:52.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:22:52.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:52.664 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:52.666 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:52.666+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:52.766 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:52.766+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:52.768 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.768 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:52.768 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:52.768 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:52.867 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:52.867+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:52.921 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:52.921 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:52.921 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:52.921 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:52.921 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:52.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:52.972 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:52.972+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:53.068 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:53.069+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:53.075 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.683107335000003, 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.076 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:53.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.683096434000007, 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:53.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:53.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.683086025999998, 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.085 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:53.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:53.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.683078085000005, 2024-09-21T14:22:53.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606340099999997, 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:53.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:53.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606321690000001, 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:53.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:53.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:53.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606295940999999, 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:53.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606274912000003, 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.108 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:53.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:53.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:53.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606249062000003, 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:53.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:53.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606230582999999, 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:53.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:53.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:53.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606212460999998, 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:53.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606192098999998, 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:53.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.130 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606172407000003, 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:53.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606155428999998, 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:53.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606135757000004, 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:53.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606106294, 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606064047999993, 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:53.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:53.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.60604506, 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.606021702000007, 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.161 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 83.605998349000004, 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:53.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:53.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:53.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:53.168 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:22:53.208 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:53.208+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:53.269 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:53.269+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:53.311 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:53.312 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:53.320 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:53.405 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:53.405+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:53.455 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:53.465 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:53.465 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:53.465 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:53.465 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:53.466 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:53.472 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:53.472+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:53.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:53.572 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:53.572+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:53.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:53.592 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:53.593 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:53.608 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:53.608 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:53.622 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.079773902, 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:53.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:53.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:53.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:53.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234197539999997, 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234183508000001, 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.633 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:53.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234167509000002, 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:53.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234152613000006, 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:53.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:53.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:53.645 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234137677000007, 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:53.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234122697000004, 2024-09-21T14:22:53.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234108081000002, 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.653 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234093033999997, 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.657 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:53.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:53.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234078475000004, 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.661 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:53.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:53.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.664 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234063750000004, 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.668 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234049310000003, 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234034960000002, 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.672 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:53.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:53.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.234014893999998, 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:53.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:53.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.233999983999993, 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.233985043999994, 2024-09-21T14:22:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:53.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:53.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.233965350999995, 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:53.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.586484462000001, 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:22:53.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.476335279000001, 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.695 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:22:53.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.572772735000001, 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:53.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:22:53.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:53.706 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:53.707 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:53.708 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:53.708 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:53.709 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:53.709 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:53.710 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:53.711 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:53.708+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:53.808 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:53.808+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:53.824 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:53.824 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:53.833 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:53.909 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:53.909+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:53.935 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:53.936 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:53.944 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:54.010 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.010+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.046 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.047 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:54.110 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:54.110+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:54.165 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.165 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.176 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:54.211 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.211+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.304 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:54.305 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:22:54.305 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-21T14:22:54.305 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:22:54.305 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:22:54.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-21T14:22:54.312 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:54.312+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:54.326 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.326 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.336 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:54.409 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:54.409 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "" 2024-09-21T14:22:54.409 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:54.412 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.413+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.479 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.479 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:54.513 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.513+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.589 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.590 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:54.613 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.613+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.700 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.700 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.709 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:54.752 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:54.752+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:54.810 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.810 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.814 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:54.815+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:54.819 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:54.915 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:54.916+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:54.962 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:54.962 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:54.971 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:55.016 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:55.017+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:55.074 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.074 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:55.126 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:55.126+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:55.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:22:55.196 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.196 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:55.247 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:55.247+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:55.318 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:55.318+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:55.348 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.348 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:55.443 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:55.443+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:55.503 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.503 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:55.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:55.520+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:55.621 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:55.621+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:55.654 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.654 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:55.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:22:55.755 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:55.755+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:55.767 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:55.767 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:55.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:55.822 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:55.822+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:55.831 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:55.832 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:55.849 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:55.849 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:55.865 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:55.865 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:55.881 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:55.881 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:55.897 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:55.897 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:55.913 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:55.914 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:55.921 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:55.921 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:55.921 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:55.921 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:55.921 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:55.923 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:55.924+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:55.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:55.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:55.931 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:55.946 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:55.947 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:56.024 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:56.024+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:56.050 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.050 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 82.828786613999995, 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.051 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:22:56.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:22:56.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336962151999998, 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.056 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336944447999997, 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.064 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336928776999997, 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336913263, 2024-09-21T14:22:56.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336898196, 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:22:56.073 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:22:56.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336883000999997, 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:22:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:22:56.081 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336867894999997, 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.082 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:22:56.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336852784999998, 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:22:56.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:22:56.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336837420000002, 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:22:56.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.096 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336821960999998, 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.097 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336806928000001, 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.101 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:22:56.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:22:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.105 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336791038999998, 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:22:56.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:22:56.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.109 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336765362999998, 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:56.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:22:56.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.113 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336750078000001, 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:22:56.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336728114000003, 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.121 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.336712130999999, 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:22:56.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.286170743, 2024-09-21T14:22:56.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:56.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:22:56.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:22:56.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.822116987000001, 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:22:56.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:22:56.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:22:56.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.889206002999998, 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:22:56.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.145 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:56.147 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:22:56.147 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:56.147+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:56.247 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:56.247+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:56.302 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.302 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:56.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:56.302 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:22:56.354 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:56.354+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:56.412 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.413 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:56.413 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:56.413 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:56.413 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:22:56.448 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:56.449+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:56.549 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:56.549+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.565 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.172392204999994, 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:56.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.172381303999998, 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:22:56.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:22:56.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:22:56.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.172370896000004, 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:56.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.172362954999997, 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:22:56.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:22:56.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:56.585 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095624970000003, 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:56.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:22:56.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:22:56.589 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095606559999993, 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.590 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:22:56.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:22:56.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:22:56.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.598 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095580811000005, 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.599 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:22:56.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:22:56.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:22:56.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.607 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095559781999995, 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:56.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:22:56.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:22:56.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.615 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095533931999995, 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:22:56.616 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:22:56.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:22:56.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:56.624 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095515453000004, 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:22:56.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:22:56.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.632 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095497331000004, 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:22:56.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:22:56.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:22:56.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:56.641 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095476969000003, 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:22:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:22:56.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.649 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095457276999994, 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:56.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:22:56.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095440299000003, 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:22:56.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:22:56.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095420626999996, 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:22:56.660 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:22:56.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095391164000006, 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:22:56.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:22:56.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095348917999999, 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095329930000005, 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:56.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:56.677 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095306571999998, 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:22:56.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:22:56.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.095283218999995, 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:22:56.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:22:56.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:56.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:56.686 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:22:56.688 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:56.687+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:56.788 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:56.788+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:56.830 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.830 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:56.830 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:22:56.830 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:22:56.923 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:56.923+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:56.981 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:56.981 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:22:56.981 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:22:56.981 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:22:56.981 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:56.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:22:56.991 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:56.991+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:57.091 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:57.091+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:22:57.133 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 107.590959767, 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:57.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:22:57.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:22:57.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:22:57.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:22:57.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:22:57.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.141 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745383404999998, 2024-09-21T14:22:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:22:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:22:57.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745369373000003, 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:22:57.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:22:57.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745353374000004, 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:22:57.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745338477999994, 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.158 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745323541999994, 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:22:57.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745308562000005, 2024-09-21T14:22:57.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:22:57.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:22:57.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745293946000004, 2024-09-21T14:22:57.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:22:57.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:22:57.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745278898999999, 2024-09-21T14:22:57.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745264340000006, 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:22:57.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:22:57.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745249615000006, 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:22:57.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:22:57.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745235175000005, 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:22:57.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745220825000004, 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:22:57.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:22:57.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745200758999999, 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.195 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745185848999995, 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.196 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:22:57.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745170908999995, 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:22:57.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:22:57.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.745151215999996, 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.097670327000003, 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:22:57.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:22:57.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.987521143999999, 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.213 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:22:57.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:22:57.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.083958600000003, 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:22:57.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:22:57.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:22:57.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:22:57.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:22:57.226 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:22:57.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:22:57.227 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:57.227+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:57.328 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:57.328+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:57.339 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:57.339 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:57.348 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:22:57.428 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:57.428+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:57.492 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:57.492 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:57.492 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:57.492 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:57.492 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:57.500 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:22:57.529 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:57.529+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.604 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.836504673, 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:22:57.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:22:57.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.836428163000001, 2024-09-21T14:22:57.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:22:57.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.835618199000002, 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:22:57.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:22:57.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:22:57.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:22:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.835591141999998, 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:22:57.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:22:57.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.835142918999999, 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:22:57.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:22:57.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:22:57.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:22:57.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:22:57.628 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.835120578999998, 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:22:57.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.835078887999998, 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:22:57.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:22:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834920222999997, 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:22:57.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834887801999997, 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.645 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:22:57.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834859272999999, 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:22:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:57.655 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834838177000002, 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:22:57.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834806006999997, 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.660 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834729518000003, 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.665 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:22:57.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:22:57.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834706578000002, 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:57.674 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834689560999998, 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:22:57.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834540707999999, 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.679 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834524997999999, 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834509574999998, 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:22:57.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.693 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834377510000003, 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.694 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:22:57.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.834342452999998, 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:22:57.699 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:57.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:57.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:22:57.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:22:57.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:57.705 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:57.706 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:22:57.707 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:22:57.707+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:22:57.807 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:57.807+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:57.849 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:57.849 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:57.850 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:22:57.850 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:57.857 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:22:57.908 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:57.909+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:57.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:22:58.000 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:58.000 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:22:58.000 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:22:58.000 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:22:58.000 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:58.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:22:58.010 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:58.010+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:58.078 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:22:58.079 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:22:58.095 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:22:58.095 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:22:58.110 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:22:58.110 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:22:58.111 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:58.111+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 34.111863743999997, 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:22:58.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.615243924000001, 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:22:58.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:22:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.615218354, 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.125 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:22:58.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:22:58.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.614263965999999, 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.130 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:22:58.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:22:58.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.613949465000001, 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:22:58.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:22:58.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:22:58.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.612534211, 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.142 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.611486760999998, 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:22:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.610203346999999, 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:22:58.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.608882121000001, 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.152 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:22:58.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:58.156 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.607841186999998, 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.607589746999999, 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:22:58.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.606543997999999, 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:22:58.165 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:22:58.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.606157056000001, 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.170 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:22:58.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.605221472, 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:22:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:22:58.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.605128802999999, 2024-09-21T14:22:58.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:22:58.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.601047585, 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:22:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:22:58.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.600864753, 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:22:58.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:22:58.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.600541041, 2024-09-21T14:22:58.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:22:58.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.600419588000001, 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.200 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 24.599160989000001, 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:22:58.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:22:58.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:22:58.205 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:22:58.205 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:22:58.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:22:58.206 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:22:58.206 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:22:58.206 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:22:58.207 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:22:58.207 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:22:58.207 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:22:58.208 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:22:58.208 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:22:58.208 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:22:58.208 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:22:58.209 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:22:58.251 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:58.251+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:58.312 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:58.312+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:58.355 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:58.355 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:58.364 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:22:58.412 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:58.412+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:58.508 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:58.508 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:58.513 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:58.513+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:58.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:22:58.614 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:22:58.615+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:22:58.659 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:58.659 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:58.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:22:58.715 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:58.716+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:58.770 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:58.770 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:58.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:22:58.823 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:58.823+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:58.881 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:58.881 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:58.889 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:22:58.931 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:58.931+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:59.018 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:22:59.018+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:22:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:22:59.119 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:59.119+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:59.144 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.144 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:22:59.165 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:22:59.220 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:22:59.220+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:22:59.275 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.275 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:22:59.329 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:59.329+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:59.388 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.388 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:22:59.419 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:22:59.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:22:59.438 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:59.438+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:59.523 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:22:59.524+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:22:59.542 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.542 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:22:59.624 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:59.624+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:59.656 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.656 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:22:59.749 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:22:59.749+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:22:59.808 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.808 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:22:59.825 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:22:59.826+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:22:59.926 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:22:59.926+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:22:59.960 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:22:59.960 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:22:59.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:00.026 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:00.027+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:00.111 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:00.112 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:00.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:00.164 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:00.164+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:00.179 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:23:00.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:23:00.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:00.228 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:00.228+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:00.228 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:00.229 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:00.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:00.328 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:00.328+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:00.352 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:00.353 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:00.369 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:00.369 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:00.385 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:00.385 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:00.386 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:00.386 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:00.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:00.400 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:00.401 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:00.417 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:00.429 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:00.430+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:00.433 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:00.433 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:00.464 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:00.465 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:00.483 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:00.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:00.569 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:00.569+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:00.631 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:00.631+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:00.672 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:00.672 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:00.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:00.672 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 111.129999263, 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284422900999999, 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:23:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:23:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284408869000004, 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:00.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284392870000005, 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:00.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284377973999995, 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284363037999995, 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:00.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.697 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284348058000006, 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284333442000005, 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:00.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:00.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284318395, 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284303836000007, 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:00.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284289111000007, 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:00.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:00.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284274671000006, 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:00.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:00.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284260321000005, 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.720 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284240255, 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:00.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:00.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:00.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284225344999996, 2024-09-21T14:23:00.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284210404999996, 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.731 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:00.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.284190711999997, 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:00.735 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:00.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:00.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.636709822999997, 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.52656064, 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:00.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:00.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.622998096000003, 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.751 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:00.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:00.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:00.756 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:00.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:00.758 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:00.758+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:00.858 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:00.858+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:00.861 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:00.861 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:00.862 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:00.862 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:00.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:00.959 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:00.959+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:01.012 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.012 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:01.012 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:01.012 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:01.012 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.021 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:01.065 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:01.064+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.357481915000001, 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.125 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:01.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:01.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:01.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.357405405000002, 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:01.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:01.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:01.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:01.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.356595441000003, 2024-09-21T14:23:01.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.356568383999999, 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:01.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:01.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.356120161, 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:01.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.356097820999999, 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.356056129999999, 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:01.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:01.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:01.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355897464999998, 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.167 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355865043999998, 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:01.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:01.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.177 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355836515, 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:01.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:01.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:01.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355815419000002, 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355783248999998, 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:01.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355706759999997, 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.190 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:01.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:01.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:01.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:01.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355683820000003, 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:01.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:01.199 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355666802999998, 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:01.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355517949999999, 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:01.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.35550224, 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.208 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:01.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355486816999999, 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.212 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355354751999997, 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:01.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:01.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.355319694999999, 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:01.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.225 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:23:01.227 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:23:01.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:01.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:23:01.228 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:01.226+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:01.327 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:01.327+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:01.369 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.369 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:01.369 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:01.369 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:01.427 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:01.427+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:01.480 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.480 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:01.480 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:01.480 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:01.480 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:01.530 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:01.529+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:01.630 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:01.630+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.631273667999999, 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.134653847999999, 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:01.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.134628278000001, 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:01.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:01.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.646 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.133673890000001, 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:01.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:01.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.650 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.133359388999999, 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.131944135000001, 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:01.654 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:01.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:01.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.130896685, 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.658 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.129613271, 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.128292044999998, 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:01.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:01.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:01.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.127251111, 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:01.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.126999671, 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:01.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:01.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:01.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.125953922000001, 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.125566979999999, 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.124631396000002, 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.686 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:01.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:01.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:01.689 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.124538727000001, 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.690 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.120457509000001, 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.694 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:01.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.120274677000001, 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.698 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:01.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.702 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.119950965000001, 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.119829511999999, 2024-09-21T14:23:01.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:01.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 28.118570912999999, 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:01.712 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:01.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:01.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:01.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:01.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:01.721 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.722 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:01.730 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:01.731+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:01.826 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:01.826 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.832 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:01.832+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:01.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:01.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:01.931+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:01.936 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:01.936 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:01.936 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:01.936 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:01.937 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:01.945 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:02.033 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:02.032+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:02.089 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657536473999997, 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:02.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:02.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:02.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657511424000006, 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:02.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:02.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657493271999996, 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:02.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657470035000003, 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:02.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657448967999997, 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:02.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:02.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.112 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.112 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:02.112 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657432607999993, 2024-09-21T14:23:02.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:02.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.113 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:02.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:02.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657402328000003, 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:02.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657387189999994, 2024-09-21T14:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:02.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:02.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.125 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657367593999993, 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:02.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:02.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.129 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657358817000002, 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657346455999999, 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:02.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:02.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:02.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:02.137 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657337038999998, 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:02.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:02.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657315847999996, 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.142 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:02.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:02.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:02.145 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657298740000002, 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:02.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657281796999996, 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:02.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.153 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657265417999994, 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.154 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657224057999997, 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.158 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:02.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:02.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 92.657204376999999, 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:02.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.165 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.252315929999995, 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:02.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 9.8895450989999993, 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.177 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:02.178 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:02.179 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:02.178+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:02.189 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:23:02.189 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:23:02.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:23:02.279 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:02.278+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:02.331 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:02.331 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:02.331 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:02.332 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:02.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:02.380 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:02.380+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:02.480 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:02.480+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:02.483 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:02.483 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:02.483 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:02.483 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:02.483 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:02.491 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:02.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:02.581 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:02.580+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:02.612 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:02.613 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:02.628 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:02.628 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040373643999999, 2024-09-21T14:23:02.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.641 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040356130000006, 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040338876000007, 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:02.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.649 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040311025999998, 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:02.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:02.653 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040289422000001, 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.657 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040272423999994, 2024-09-21T14:23:02.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:02.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.662 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040255404000007, 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:02.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040237954000006, 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.667 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040220816000001, 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.671 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040197589000002, 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:02.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:02.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040181314999998, 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:02.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:02.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040156943, 2024-09-21T14:23:02.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:02.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040134375999997, 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:02.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:02.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040117455000001, 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040100956000003, 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.696 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:02.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:02.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040079958999996, 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:02.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:02.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:02.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 93.040057900999997, 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.865987028999996, 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:02.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 38.723753905000002, 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.491043828999999, 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:02.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:02.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:02.722 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:02.723 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:02.723 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:02.723 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:02.724 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:02.723+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:02.724 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:02.725 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:02.725 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:02.726 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:02.726 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:02.823 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:02.823+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:02.828 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:02.828 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:02.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:02.924 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:02.924+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:02.980 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:02.980 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:02.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:03.032 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:03.032+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:03.091 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.091 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:03.126 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:03.126+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:03.200 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:23:03.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:23:03.250 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.250 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.251 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:03.251+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:03.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:03.327 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:03.326+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:03.406 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.406 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:03.459 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:03.459+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:03.518 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.518 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:03.568 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:03.568+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:03.629 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:03.629+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:03.669 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.670 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:03.730 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:03.730+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:03.779 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.779 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:03.831 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:03.831+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:03.889 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:03.889 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:03.898 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:03.931 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:03.931+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:04.032 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:04.033+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:04.039 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.039 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.048 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:04.133 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:04.133+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:04.190 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.190 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:04.200 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:23:04.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:23:04.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:23:04.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:23:04.258 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:04.258+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:04.314 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.315 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:04.369 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:04.368+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:04.432 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.432 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.436 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:04.435+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:04.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:04.536 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:04.536+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:23:04.550 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:23:04.562 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-21T14:23:04.435938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1073055601,"num_objects":1383,"num_object_clones":324,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1383,"num_whiteouts":624,"num_read":140400,"num_read_kb":947481,"num_write":20562,"num_write_kb":505801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2826,"num_bytes_recovered":269458226,"num_keys_recovered":619,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17338,"ondisk_log_size":17338,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":290400,"kb_used_data":36300,"kb_used_omap":7,"kb_used_meta":253944,"kb_avail":471568800,"statfs":{"total":483183820800,"available":482886451200,"internally_reserved":0,"allocated":37171200,"data_stored":263471043,"data_compressed":15229112,"data_compressed_allocated":17518592,"data_compressed_original":250008252,"omap_allocated":7964,"internal_metadata":260038884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-163261297,"num_objects":-209,"num_object_clones":-88,"num_object_copies":-418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-209,"num_whiteouts":-57,"num_read":1333,"num_read_kb":1139,"num_write":235,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000628"},"pg_stats":[{"pgid":"3.1f","version":"114'336","reported_seq":3108,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.200439+0000","last_change":"2024-09-21T14:23:02.200662+0000","last_active":"2024-09-21T14:23:02.200439+0000","last_peered":"2024-09-21T14:23:02.200439+0000","last_clean":"2024-09-21T14:23:02.200439+0000","last_became_active":"2024-09-21T14:22:35.216886+0000","last_became_peered":"2024-09-21T14:22:35.216886+0000","last_unstale":"2024-09-21T14:23:02.200439+0000","last_undegraded":"2024-09-21T14:23:02.200439+0000","last_fullsized":"2024-09-21T14:23:02.200439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:49.746608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011588830000000001,"stat_sum":{"num_bytes":15177272,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":3451,"num_read_kb":17450,"num_write":319,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3807877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1e","version":"114'315","reported_seq":2101,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.199793+0000","last_change":"2024-09-21T14:23:02.199994+0000","last_active":"2024-09-21T14:23:02.199793+0000","last_peered":"2024-09-21T14:23:02.199793+0000","last_clean":"2024-09-21T14:23:02.199793+0000","last_became_active":"2024-09-21T14:22:29.158180+0000","last_became_peered":"2024-09-21T14:22:29.158180+0000","last_unstale":"2024-09-21T14:23:02.199793+0000","last_undegraded":"2024-09-21T14:23:02.199793+0000","last_fullsized":"2024-09-21T14:23:02.199793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:47.183599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056511400000000002,"stat_sum":{"num_bytes":27622786,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2107,"num_read_kb":53406,"num_write":312,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":32363151,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1d","version":"112'414","reported_seq":3314,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196844+0000","last_change":"2024-09-21T14:23:02.197039+0000","last_active":"2024-09-21T14:23:02.196844+0000","last_peered":"2024-09-21T14:23:02.196844+0000","last_clean":"2024-09-21T14:23:02.196844+0000","last_became_active":"2024-09-21T14:22:36.210300+0000","last_became_peered":"2024-09-21T14:22:36.210300+0000","last_unstale":"2024-09-21T14:23:02.196844+0000","last_undegraded":"2024-09-21T14:23:02.196844+0000","last_fullsized":"2024-09-21T14:23:02.196844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:16.513441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001651975,"stat_sum":{"num_bytes":11488523,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":3927,"num_read_kb":20085,"num_write":428,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14805334,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1c","version":"115'436","reported_seq":5917,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.250639+0000","last_change":"2024-09-21T14:23:02.196908+0000","last_active":"2024-09-21T14:23:02.250639+0000","last_peered":"2024-09-21T14:23:02.250639+0000","last_clean":"2024-09-21T14:23:02.250639+0000","last_became_active":"2024-09-21T14:22:35.214138+0000","last_became_peered":"2024-09-21T14:22:35.214138+0000","last_unstale":"2024-09-21T14:23:02.250639+0000","last_undegraded":"2024-09-21T14:23:02.250639+0000","last_fullsized":"2024-09-21T14:23:02.250639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.144123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079420999999999995,"stat_sum":{"num_bytes":42682794,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":6331,"num_read_kb":82952,"num_write":475,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":26488863,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1b","version":"115'507","reported_seq":4684,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.280752+0000","last_change":"2024-09-21T14:23:02.193400+0000","last_active":"2024-09-21T14:23:02.280752+0000","last_peered":"2024-09-21T14:23:02.280752+0000","last_clean":"2024-09-21T14:23:02.280752+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:23:02.280752+0000","last_undegraded":"2024-09-21T14:23:02.280752+0000","last_fullsized":"2024-09-21T14:23:02.280752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485280000000001,"stat_sum":{"num_bytes":60930249,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":6272,"num_read_kb":42754,"num_write":434,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1a","version":"115'500","reported_seq":5197,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.315446+0000","last_change":"2024-09-21T14:23:02.199795+0000","last_active":"2024-09-21T14:23:02.315446+0000","last_peered":"2024-09-21T14:23:02.315446+0000","last_clean":"2024-09-21T14:23:02.315446+0000","last_became_active":"2024-09-21T14:22:35.215561+0000","last_became_peered":"2024-09-21T14:22:35.215561+0000","last_unstale":"2024-09-21T14:23:02.315446+0000","last_undegraded":"2024-09-21T14:23:02.315446+0000","last_fullsized":"2024-09-21T14:23:02.315446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:05.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058984800000000002,"stat_sum":{"num_bytes":27181375,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":6512,"num_read_kb":65589,"num_write":539,"num_write_kb":34390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":29327918,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.19","version":"115'777","reported_seq":5674,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.313876+0000","last_change":"2024-09-21T14:23:02.193776+0000","last_active":"2024-09-21T14:23:02.313876+0000","last_peered":"2024-09-21T14:23:02.313876+0000","last_clean":"2024-09-21T14:23:02.313876+0000","last_became_active":"2024-09-21T14:22:35.217004+0000","last_became_peered":"2024-09-21T14:22:35.217004+0000","last_unstale":"2024-09-21T14:23:02.313876+0000","last_undegraded":"2024-09-21T14:23:02.313876+0000","last_fullsized":"2024-09-21T14:23:02.313876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:44.918657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013530339999999999,"stat_sum":{"num_bytes":27514025,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":7350,"num_read_kb":30193,"num_write":1050,"num_write_kb":11024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":7435469,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.18","version":"115'274","reported_seq":1928,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.315922+0000","last_change":"2024-09-21T14:23:02.199748+0000","last_active":"2024-09-21T14:23:02.315922+0000","last_peered":"2024-09-21T14:23:02.315922+0000","last_clean":"2024-09-21T14:23:02.315922+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:23:02.315922+0000","last_undegraded":"2024-09-21T14:23:02.315922+0000","last_fullsized":"2024-09-21T14:23:02.315922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059072300000000003,"stat_sum":{"num_bytes":13244526,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":1650,"num_read_kb":12534,"num_write":235,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.17","version":"115'752","reported_seq":3123,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.276331+0000","last_change":"2024-09-21T14:23:02.200635+0000","last_active":"2024-09-21T14:23:02.276331+0000","last_peered":"2024-09-21T14:23:02.276331+0000","last_clean":"2024-09-21T14:23:02.276331+0000","last_became_active":"2024-09-21T14:22:35.217012+0000","last_became_peered":"2024-09-21T14:22:35.217012+0000","last_unstale":"2024-09-21T14:23:02.276331+0000","last_undegraded":"2024-09-21T14:23:02.276331+0000","last_fullsized":"2024-09-21T14:23:02.276331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:02.683415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744870000000001,"stat_sum":{"num_bytes":11195071,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4771,"num_read_kb":10236,"num_write":940,"num_write_kb":13984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":5304982,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.16","version":"113'488","reported_seq":2649,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193148+0000","last_change":"2024-09-21T14:23:02.193312+0000","last_active":"2024-09-21T14:23:02.193148+0000","last_peered":"2024-09-21T14:23:02.193148+0000","last_clean":"2024-09-21T14:23:02.193148+0000","last_became_active":"2024-09-21T14:22:29.161682+0000","last_became_peered":"2024-09-21T14:22:29.161682+0000","last_unstale":"2024-09-21T14:23:02.193148+0000","last_undegraded":"2024-09-21T14:23:02.193148+0000","last_fullsized":"2024-09-21T14:23:02.193148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:15.765317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098179499999999993,"stat_sum":{"num_bytes":7956855,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":3237,"num_read_kb":13890,"num_write":637,"num_write_kb":7256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":3589748,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.15","version":"115'751","reported_seq":2740,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196555+0000","last_change":"2024-09-21T14:23:02.196555+0000","last_active":"2024-09-21T14:23:02.196555+0000","last_peered":"2024-09-21T14:23:02.196555+0000","last_clean":"2024-09-21T14:23:02.196555+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:23:02.196555+0000","last_undegraded":"2024-09-21T14:23:02.196555+0000","last_fullsized":"2024-09-21T14:23:02.196555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050380200000000003,"stat_sum":{"num_bytes":37470362,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":4828,"num_read_kb":24612,"num_write":1036,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.14","version":"115'439","reported_seq":3154,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196786+0000","last_change":"2024-09-21T14:23:02.196786+0000","last_active":"2024-09-21T14:23:02.196786+0000","last_peered":"2024-09-21T14:23:02.196786+0000","last_clean":"2024-09-21T14:23:02.196786+0000","last_became_active":"2024-09-21T14:22:29.164131+0000","last_became_peered":"2024-09-21T14:22:29.164131+0000","last_unstale":"2024-09-21T14:23:02.196786+0000","last_undegraded":"2024-09-21T14:23:02.196786+0000","last_fullsized":"2024-09-21T14:23:02.196786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:06.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054163799999999999,"stat_sum":{"num_bytes":18588570,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":27,"num_read":4846,"num_read_kb":16747,"num_write":471,"num_write_kb":9905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":6514761,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146990+0000","last_change":"2024-09-21T14:22:32.174490+0000","last_active":"2024-09-21T14:23:01.146990+0000","last_peered":"2024-09-21T14:23:01.146990+0000","last_clean":"2024-09-21T14:23:01.146990+0000","last_became_active":"2024-09-21T14:22:32.174175+0000","last_became_peered":"2024-09-21T14:22:32.174175+0000","last_unstale":"2024-09-21T14:23:01.146990+0000","last_undegraded":"2024-09-21T14:23:01.146990+0000","last_fullsized":"2024-09-21T14:23:01.146990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:54.846715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"113'421","reported_seq":2965,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187447+0000","last_change":"2024-09-21T14:23:02.187650+0000","last_active":"2024-09-21T14:23:02.187447+0000","last_peered":"2024-09-21T14:23:02.187447+0000","last_clean":"2024-09-21T14:23:02.187447+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:23:02.187447+0000","last_undegraded":"2024-09-21T14:23:02.187447+0000","last_fullsized":"2024-09-21T14:23:02.187447+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066630900000000002,"stat_sum":{"num_bytes":38840074,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":3504,"num_read_kb":12859,"num_write":395,"num_write_kb":5718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146884+0000","last_change":"2024-09-21T14:22:26.141620+0000","last_active":"2024-09-21T14:23:01.146884+0000","last_peered":"2024-09-21T14:23:01.146884+0000","last_clean":"2024-09-21T14:23:01.146884+0000","last_became_active":"2024-09-21T14:22:26.141314+0000","last_became_peered":"2024-09-21T14:22:26.141314+0000","last_unstale":"2024-09-21T14:23:01.146884+0000","last_undegraded":"2024-09-21T14:23:01.146884+0000","last_fullsized":"2024-09-21T14:23:01.146884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:24:21.823458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"115'479","reported_seq":3029,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187569+0000","last_change":"2024-09-21T14:23:02.187569+0000","last_active":"2024-09-21T14:23:02.187569+0000","last_peered":"2024-09-21T14:23:02.187569+0000","last_clean":"2024-09-21T14:23:02.187569+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:23:02.187569+0000","last_undegraded":"2024-09-21T14:23:02.187569+0000","last_fullsized":"2024-09-21T14:23:02.187569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000612033,"stat_sum":{"num_bytes":27183207,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":3877,"num_read_kb":18030,"num_write":592,"num_write_kb":5955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146714+0000","last_change":"2024-09-21T14:22:26.141948+0000","last_active":"2024-09-21T14:23:01.146714+0000","last_peered":"2024-09-21T14:23:01.146714+0000","last_clean":"2024-09-21T14:23:01.146714+0000","last_became_active":"2024-09-21T14:22:26.141693+0000","last_became_peered":"2024-09-21T14:22:26.141693+0000","last_unstale":"2024-09-21T14:23:01.146714+0000","last_undegraded":"2024-09-21T14:23:01.146714+0000","last_fullsized":"2024-09-21T14:23:01.146714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:29:48.520733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"115'375","reported_seq":3089,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193514+0000","last_change":"2024-09-21T14:23:02.193514+0000","last_active":"2024-09-21T14:23:02.193514+0000","last_peered":"2024-09-21T14:23:02.193514+0000","last_clean":"2024-09-21T14:23:02.193514+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:23:02.193514+0000","last_undegraded":"2024-09-21T14:23:02.193514+0000","last_fullsized":"2024-09-21T14:23:02.193514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766750000000001,"stat_sum":{"num_bytes":40851495,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":2834,"num_read_kb":36189,"num_write":346,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146543+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:23:01.146543+0000","last_peered":"2024-09-21T14:23:01.146543+0000","last_clean":"2024-09-21T14:23:01.146543+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:23:01.146543+0000","last_undegraded":"2024-09-21T14:23:01.146543+0000","last_fullsized":"2024-09-21T14:23:01.146543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"115'512","reported_seq":5406,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193596+0000","last_change":"2024-09-21T14:23:02.193596+0000","last_active":"2024-09-21T14:23:02.193596+0000","last_peered":"2024-09-21T14:23:02.193596+0000","last_clean":"2024-09-21T14:23:02.193596+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:23:02.193596+0000","last_undegraded":"2024-09-21T14:23:02.193596+0000","last_fullsized":"2024-09-21T14:23:02.193596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001210785,"stat_sum":{"num_bytes":42346199,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":6590,"num_read_kb":31308,"num_write":503,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1","version":"114'304","reported_seq":2558,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187730+0000","last_change":"2024-09-21T14:23:02.187953+0000","last_active":"2024-09-21T14:23:02.187730+0000","last_peered":"2024-09-21T14:23:02.187730+0000","last_clean":"2024-09-21T14:23:02.187730+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:23:02.187730+0000","last_undegraded":"2024-09-21T14:23:02.187730+0000","last_fullsized":"2024-09-21T14:23:02.187730+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095413400000000004,"stat_sum":{"num_bytes":36636910,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":3088,"num_read_kb":21572,"num_write":270,"num_write_kb":9367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"2.0","version":"17'117","reported_seq":216,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.144044+0000","last_change":"2024-09-21T14:22:29.157860+0000","last_active":"2024-09-21T14:23:01.144044+0000","last_peered":"2024-09-21T14:23:01.144044+0000","last_clean":"2024-09-21T14:23:01.144044+0000","last_became_active":"2024-09-21T14:22:29.157461+0000","last_became_peered":"2024-09-21T14:22:29.157461+0000","last_unstale":"2024-09-21T14:23:01.144044+0000","last_undegraded":"2024-09-21T14:23:01.144044+0000","last_fullsized":"2024-09-21T14:23:01.144044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:27.251019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":127,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.151401+0000","last_change":"2024-09-21T14:22:32.181750+0000","last_active":"2024-09-21T14:23:01.151401+0000","last_peered":"2024-09-21T14:23:01.151401+0000","last_clean":"2024-09-21T14:23:01.151401+0000","last_became_active":"2024-09-21T14:22:32.177114+0000","last_became_peered":"2024-09-21T14:22:32.177114+0000","last_unstale":"2024-09-21T14:23:01.151401+0000","last_undegraded":"2024-09-21T14:23:01.151401+0000","last_fullsized":"2024-09-21T14:23:01.151401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:00.138436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"115'747","reported_seq":3273,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.269616+0000","last_change":"2024-09-21T14:23:02.199483+0000","last_active":"2024-09-21T14:23:02.269616+0000","last_peered":"2024-09-21T14:23:02.269616+0000","last_clean":"2024-09-21T14:23:02.269616+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:23:02.269616+0000","last_undegraded":"2024-09-21T14:23:02.269616+0000","last_fullsized":"2024-09-21T14:23:02.269616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037525299999999998,"stat_sum":{"num_bytes":24563309,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":5340,"num_read_kb":124688,"num_write":970,"num_write_kb":60251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.2","version":"17'2","reported_seq":116,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.155641+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:23:01.155641+0000","last_peered":"2024-09-21T14:23:01.155641+0000","last_clean":"2024-09-21T14:23:01.155641+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:23:01.155641+0000","last_undegraded":"2024-09-21T14:23:01.155641+0000","last_fullsized":"2024-09-21T14:23:01.155641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"115'1069","reported_seq":2489,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.286134+0000","last_change":"2024-09-21T14:23:02.187345+0000","last_active":"2024-09-21T14:23:02.286134+0000","last_peered":"2024-09-21T14:23:02.286134+0000","last_clean":"2024-09-21T14:23:02.286134+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:23:02.286134+0000","last_undegraded":"2024-09-21T14:23:02.286134+0000","last_fullsized":"2024-09-21T14:23:02.286134+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1069,"log_dups_size":0,"ondisk_log_size":1069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000411691,"stat_sum":{"num_bytes":17741403,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":4577,"num_read_kb":16301,"num_write":1532,"num_write_kb":8571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.143777+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:23:01.143777+0000","last_peered":"2024-09-21T14:23:01.143777+0000","last_clean":"2024-09-21T14:23:01.143777+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:23:01.143777+0000","last_undegraded":"2024-09-21T14:23:01.143777+0000","last_fullsized":"2024-09-21T14:23:01.143777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"114'533","reported_seq":2544,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192529+0000","last_change":"2024-09-21T14:23:02.192692+0000","last_active":"2024-09-21T14:23:02.192529+0000","last_peered":"2024-09-21T14:23:02.192529+0000","last_clean":"2024-09-21T14:23:02.192529+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:23:02.192529+0000","last_undegraded":"2024-09-21T14:23:02.192529+0000","last_fullsized":"2024-09-21T14:23:02.192529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053585799999999995,"stat_sum":{"num_bytes":62534416,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":3545,"num_read_kb":12364,"num_write":654,"num_write_kb":5627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.143953+0000","last_change":"2024-09-21T14:22:32.172034+0000","last_active":"2024-09-21T14:23:01.143953+0000","last_peered":"2024-09-21T14:23:01.143953+0000","last_clean":"2024-09-21T14:23:01.143953+0000","last_became_active":"2024-09-21T14:22:32.171774+0000","last_became_peered":"2024-09-21T14:22:32.171774+0000","last_unstale":"2024-09-21T14:23:01.143953+0000","last_undegraded":"2024-09-21T14:23:01.143953+0000","last_fullsized":"2024-09-21T14:23:01.143953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:41.624554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"113'593","reported_seq":3342,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192495+0000","last_change":"2024-09-21T14:23:02.192495+0000","last_active":"2024-09-21T14:23:02.192495+0000","last_peered":"2024-09-21T14:23:02.192495+0000","last_clean":"2024-09-21T14:23:02.192495+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:23:02.192495+0000","last_undegraded":"2024-09-21T14:23:02.192495+0000","last_fullsized":"2024-09-21T14:23:02.192495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043803100000000002,"stat_sum":{"num_bytes":53764636,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":5407,"num_read_kb":15634,"num_write":664,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.9","version":"115'610","reported_seq":4329,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.290206+0000","last_change":"2024-09-21T14:23:02.192888+0000","last_active":"2024-09-21T14:23:02.290206+0000","last_peered":"2024-09-21T14:23:02.290206+0000","last_clean":"2024-09-21T14:23:02.290206+0000","last_became_active":"2024-09-21T14:22:29.166889+0000","last_became_peered":"2024-09-21T14:22:29.166889+0000","last_unstale":"2024-09-21T14:23:02.290206+0000","last_undegraded":"2024-09-21T14:23:02.290206+0000","last_fullsized":"2024-09-21T14:23:02.290206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:48.958859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066009,"stat_sum":{"num_bytes":70625345,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":7151,"num_read_kb":84194,"num_write":658,"num_write_kb":55675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":51428954,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.a","version":"115'416","reported_seq":1815,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.202462+0000","last_change":"2024-09-21T14:23:02.202462+0000","last_active":"2024-09-21T14:23:02.202462+0000","last_peered":"2024-09-21T14:23:02.202462+0000","last_clean":"2024-09-21T14:23:02.202462+0000","last_became_active":"2024-09-21T14:22:35.217329+0000","last_became_peered":"2024-09-21T14:22:35.217329+0000","last_unstale":"2024-09-21T14:23:02.202462+0000","last_undegraded":"2024-09-21T14:23:02.202462+0000","last_fullsized":"2024-09-21T14:23:02.202462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:17:58.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027907600000000001,"stat_sum":{"num_bytes":13318421,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":21,"num_read":1268,"num_read_kb":8559,"num_write":404,"num_write_kb":7229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":11062623,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.b","version":"111'436","reported_seq":2332,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.202340+0000","last_change":"2024-09-21T14:23:02.202487+0000","last_active":"2024-09-21T14:23:02.202340+0000","last_peered":"2024-09-21T14:23:02.202340+0000","last_clean":"2024-09-21T14:23:02.202340+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:23:02.202340+0000","last_undegraded":"2024-09-21T14:23:02.202340+0000","last_fullsized":"2024-09-21T14:23:02.202340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026751500000000002,"stat_sum":{"num_bytes":33366655,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":2687,"num_read_kb":10392,"num_write":542,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.c","version":"114'664","reported_seq":2231,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196243+0000","last_change":"2024-09-21T14:23:02.196567+0000","last_active":"2024-09-21T14:23:02.196243+0000","last_peered":"2024-09-21T14:23:02.196243+0000","last_clean":"2024-09-21T14:23:02.196243+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:23:02.196243+0000","last_undegraded":"2024-09-21T14:23:02.196243+0000","last_fullsized":"2024-09-21T14:23:02.196243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059781400000000001,"stat_sum":{"num_bytes":25328278,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3658,"num_read_kb":17839,"num_write":859,"num_write_kb":10999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.d","version":"113'506","reported_seq":2720,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195490+0000","last_change":"2024-09-21T14:23:02.196205+0000","last_active":"2024-09-21T14:23:02.195490+0000","last_peered":"2024-09-21T14:23:02.195490+0000","last_clean":"2024-09-21T14:23:02.195490+0000","last_became_active":"2024-09-21T14:22:36.212777+0000","last_became_peered":"2024-09-21T14:22:36.212777+0000","last_unstale":"2024-09-21T14:23:02.195490+0000","last_undegraded":"2024-09-21T14:23:02.195490+0000","last_fullsized":"2024-09-21T14:23:02.195490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:54:27.444035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079686199999999996,"stat_sum":{"num_bytes":18872961,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":3373,"num_read_kb":14313,"num_write":668,"num_write_kb":16264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5387372,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.e","version":"110'940","reported_seq":3644,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192921+0000","last_change":"2024-09-21T14:23:02.193127+0000","last_active":"2024-09-21T14:23:02.192921+0000","last_peered":"2024-09-21T14:23:02.192921+0000","last_clean":"2024-09-21T14:23:02.192921+0000","last_became_active":"2024-09-21T14:22:23.539499+0000","last_became_peered":"2024-09-21T14:22:23.539499+0000","last_unstale":"2024-09-21T14:23:02.192921+0000","last_undegraded":"2024-09-21T14:23:02.192921+0000","last_fullsized":"2024-09-21T14:23:02.192921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:00:47.787223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089003300000000004,"stat_sum":{"num_bytes":58911077,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":6047,"num_read_kb":20604,"num_write":1393,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":7784436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.f","version":"115'450","reported_seq":4833,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195790+0000","last_change":"2024-09-21T14:23:02.195790+0000","last_active":"2024-09-21T14:23:02.195790+0000","last_peered":"2024-09-21T14:23:02.195790+0000","last_clean":"2024-09-21T14:23:02.195790+0000","last_became_active":"2024-09-21T14:22:36.216864+0000","last_became_peered":"2024-09-21T14:22:36.216864+0000","last_unstale":"2024-09-21T14:23:02.195790+0000","last_undegraded":"2024-09-21T14:23:02.195790+0000","last_fullsized":"2024-09-21T14:23:02.195790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:43.577416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036727999999999999,"stat_sum":{"num_bytes":63492156,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":5787,"num_read_kb":25840,"num_write":438,"num_write_kb":21462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6327001,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.10","version":"115'378","reported_seq":2974,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.269389+0000","last_change":"2024-09-21T14:23:02.199468+0000","last_active":"2024-09-21T14:23:02.269389+0000","last_peered":"2024-09-21T14:23:02.269389+0000","last_clean":"2024-09-21T14:23:02.269389+0000","last_became_active":"2024-09-21T14:22:35.208525+0000","last_became_peered":"2024-09-21T14:22:35.208525+0000","last_unstale":"2024-09-21T14:23:02.269389+0000","last_undegraded":"2024-09-21T14:23:02.269389+0000","last_fullsized":"2024-09-21T14:23:02.269389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.874702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017700000000001,"stat_sum":{"num_bytes":36630188,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":2775,"num_read_kb":16167,"num_write":345,"num_write_kb":7253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":5015729,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.11","version":"111'331","reported_seq":2516,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187780+0000","last_change":"2024-09-21T14:23:02.188002+0000","last_active":"2024-09-21T14:23:02.187780+0000","last_peered":"2024-09-21T14:23:02.187780+0000","last_clean":"2024-09-21T14:23:02.187780+0000","last_became_active":"2024-09-21T14:22:35.208396+0000","last_became_peered":"2024-09-21T14:22:35.208396+0000","last_unstale":"2024-09-21T14:23:02.187780+0000","last_undegraded":"2024-09-21T14:23:02.187780+0000","last_fullsized":"2024-09-21T14:23:02.187780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:35.143804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096473299999999995,"stat_sum":{"num_bytes":55838171,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":1890,"num_read_kb":23055,"num_write":386,"num_write_kb":18874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":22976540,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.12","version":"114'481","reported_seq":3147,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.301062+0000","last_change":"2024-09-21T14:23:02.187351+0000","last_active":"2024-09-21T14:23:02.301062+0000","last_peered":"2024-09-21T14:23:02.301062+0000","last_clean":"2024-09-21T14:23:02.301062+0000","last_became_active":"2024-09-21T14:22:29.157244+0000","last_became_peered":"2024-09-21T14:22:29.157244+0000","last_unstale":"2024-09-21T14:23:02.301062+0000","last_undegraded":"2024-09-21T14:23:02.301062+0000","last_fullsized":"2024-09-21T14:23:02.301062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:24.971948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042565000000000002,"stat_sum":{"num_bytes":26411215,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":4521,"num_read_kb":22673,"num_write":525,"num_write_kb":10769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5431126,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.13","version":"114'984","reported_seq":4285,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195855+0000","last_change":"2024-09-21T14:23:02.196153+0000","last_active":"2024-09-21T14:23:02.195855+0000","last_peered":"2024-09-21T14:23:02.195855+0000","last_clean":"2024-09-21T14:23:02.195855+0000","last_became_active":"2024-09-21T14:22:41.248710+0000","last_became_peered":"2024-09-21T14:22:41.248710+0000","last_unstale":"2024-09-21T14:23:02.195855+0000","last_undegraded":"2024-09-21T14:23:02.195855+0000","last_fullsized":"2024-09-21T14:23:02.195855+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":984,"log_dups_size":0,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:41.665719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040070399999999999,"stat_sum":{"num_bytes":24156690,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":7083,"num_read_kb":24352,"num_write":1381,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":13233521,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1072465214,"num_objects":1379,"num_object_clones":324,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":624,"num_read":140284,"num_read_kb":947381,"num_write":20401,"num_write_kb":503383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2823,"num_bytes_recovered":268867858,"num_keys_recovered":619,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57163776,"data_stored":531111414,"data_compressed":30607838,"data_compressed_allocated":36036608,"data_compressed_original":513813665,"omap_allocated":0,"internal_metadata":0},"log_size":17218,"ondisk_log_size":17218,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98865,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55540,"kb_used_data":6376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94316300,"statfs":{"total":96636764160,"available":96579891200,"internally_reserved":0,"allocated":6529024,"data_stored":41922665,"data_compressed":2249393,"data_compressed_allocated":2801664,"data_compressed_original":39478747,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.25600000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.441},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34332,"kb_used_data":7308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337508,"statfs":{"total":96636764160,"available":96601608192,"internally_reserved":0,"allocated":7483392,"data_stored":55841343,"data_compressed":3409624,"data_compressed_allocated":3903488,"data_compressed_original":53456152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.41199999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.437}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.27900000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.31}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56188,"kb_used_data":7016,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315652,"statfs":{"total":96636764160,"available":96579227648,"internally_reserved":0,"allocated":7184384,"data_stored":49301935,"data_compressed":2889364,"data_compressed_allocated":3411968,"data_compressed_original":46672523,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.7430000000000003,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.39000000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.32600000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.378}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61436,"kb_used_data":8144,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94310404,"statfs":{"total":96636764160,"available":96573853696,"internally_reserved":0,"allocated":8339456,"data_stored":58491366,"data_compressed":3384302,"data_compressed_allocated":3784704,"data_compressed_original":55290196,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.32000000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.38200000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.35099999999999998}]}]},{"osd":3,"up_from":0,"seq":55834574874,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574873,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":0,"seq":55834574874,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82904,"kb_used_data":7456,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94288936,"statfs":{"total":96636764160,"available":96551870464,"internally_reserved":0,"allocated":7634944,"data_stored":57913734,"data_compressed":3296429,"data_compressed_allocated":3616768,"data_compressed_original":55110634,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.246}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.28799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.29999999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.44},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.38100000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6127616,"data_stored":56100424,"data_compressed":3216946,"data_compressed_allocated":3506176,"data_compressed_original":53963754,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12337152,"data_stored":119058163,"data_compressed":6888762,"data_compressed_allocated":8253440,"data_compressed_original":115661228,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":140569390,"data_compressed":7948729,"data_compressed_allocated":9515008,"data_compressed_original":137234462,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6778880,"data_stored":56650639,"data_compressed":3306180,"data_compressed_allocated":3678208,"data_compressed_original":54171988,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5750784,"data_stored":48055388,"data_compressed":2856954,"data_compressed_allocated":3366912,"data_compressed_original":46099083,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5763072,"data_stored":53351403,"data_compressed":3253300,"data_compressed_allocated":3706880,"data_compressed_original":51735832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5144576,"data_stored":41251292,"data_compressed":2248801,"data_compressed_allocated":2797568,"data_compressed_original":39466459,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:23:04.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:23:04.583 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.583 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:04.647 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:04.647+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:04.692 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.692 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.701 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:04.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:04.739 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:04.739+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:04.803 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.803 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.812 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:04.839 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:04.839+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:04.856 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:04.856 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:04.872 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:04.872 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:04.886 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:23:04.887 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:23:04.893 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:04.899 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-21T14:23:04.435938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1073055601,"num_objects":1383,"num_object_clones":324,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1383,"num_whiteouts":624,"num_read":140400,"num_read_kb":947481,"num_write":20562,"num_write_kb":505801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2826,"num_bytes_recovered":269458226,"num_keys_recovered":619,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17338,"ondisk_log_size":17338,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":290400,"kb_used_data":36300,"kb_used_omap":7,"kb_used_meta":253944,"kb_avail":471568800,"statfs":{"total":483183820800,"available":482886451200,"internally_reserved":0,"allocated":37171200,"data_stored":263471043,"data_compressed":15229112,"data_compressed_allocated":17518592,"data_compressed_original":250008252,"omap_allocated":7964,"internal_metadata":260038884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-163261297,"num_objects":-209,"num_object_clones":-88,"num_object_copies":-418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-209,"num_whiteouts":-57,"num_read":1333,"num_read_kb":1139,"num_write":235,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000628"},"pg_stats":[{"pgid":"3.1f","version":"114'336","reported_seq":3108,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.200439+0000","last_change":"2024-09-21T14:23:02.200662+0000","last_active":"2024-09-21T14:23:02.200439+0000","last_peered":"2024-09-21T14:23:02.200439+0000","last_clean":"2024-09-21T14:23:02.200439+0000","last_became_active":"2024-09-21T14:22:35.216886+0000","last_became_peered":"2024-09-21T14:22:35.216886+0000","last_unstale":"2024-09-21T14:23:02.200439+0000","last_undegraded":"2024-09-21T14:23:02.200439+0000","last_fullsized":"2024-09-21T14:23:02.200439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:49.746608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011588830000000001,"stat_sum":{"num_bytes":15177272,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":3451,"num_read_kb":17450,"num_write":319,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3807877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1e","version":"114'315","reported_seq":2101,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.199793+0000","last_change":"2024-09-21T14:23:02.199994+0000","last_active":"2024-09-21T14:23:02.199793+0000","last_peered":"2024-09-21T14:23:02.199793+0000","last_clean":"2024-09-21T14:23:02.199793+0000","last_became_active":"2024-09-21T14:22:29.158180+0000","last_became_peered":"2024-09-21T14:22:29.158180+0000","last_unstale":"2024-09-21T14:23:02.199793+0000","last_undegraded":"2024-09-21T14:23:02.199793+0000","last_fullsized":"2024-09-21T14:23:02.199793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:47.183599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056511400000000002,"stat_sum":{"num_bytes":27622786,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2107,"num_read_kb":53406,"num_write":312,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":32363151,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1d","version":"112'414","reported_seq":3314,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196844+0000","last_change":"2024-09-21T14:23:02.197039+0000","last_active":"2024-09-21T14:23:02.196844+0000","last_peered":"2024-09-21T14:23:02.196844+0000","last_clean":"2024-09-21T14:23:02.196844+0000","last_became_active":"2024-09-21T14:22:36.210300+0000","last_became_peered":"2024-09-21T14:22:36.210300+0000","last_unstale":"2024-09-21T14:23:02.196844+0000","last_undegraded":"2024-09-21T14:23:02.196844+0000","last_fullsized":"2024-09-21T14:23:02.196844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:16.513441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001651975,"stat_sum":{"num_bytes":11488523,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":3927,"num_read_kb":20085,"num_write":428,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14805334,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1c","version":"115'436","reported_seq":5917,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.250639+0000","last_change":"2024-09-21T14:23:02.196908+0000","last_active":"2024-09-21T14:23:02.250639+0000","last_peered":"2024-09-21T14:23:02.250639+0000","last_clean":"2024-09-21T14:23:02.250639+0000","last_became_active":"2024-09-21T14:22:35.214138+0000","last_became_peered":"2024-09-21T14:22:35.214138+0000","last_unstale":"2024-09-21T14:23:02.250639+0000","last_undegraded":"2024-09-21T14:23:02.250639+0000","last_fullsized":"2024-09-21T14:23:02.250639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.144123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079420999999999995,"stat_sum":{"num_bytes":42682794,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":6331,"num_read_kb":82952,"num_write":475,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":26488863,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1b","version":"115'507","reported_seq":4684,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.280752+0000","last_change":"2024-09-21T14:23:02.193400+0000","last_active":"2024-09-21T14:23:02.280752+0000","last_peered":"2024-09-21T14:23:02.280752+0000","last_clean":"2024-09-21T14:23:02.280752+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:23:02.280752+0000","last_undegraded":"2024-09-21T14:23:02.280752+0000","last_fullsized":"2024-09-21T14:23:02.280752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485280000000001,"stat_sum":{"num_bytes":60930249,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":6272,"num_read_kb":42754,"num_write":434,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1a","version":"115'500","reported_seq":5197,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.315446+0000","last_change":"2024-09-21T14:23:02.199795+0000","last_active":"2024-09-21T14:23:02.315446+0000","last_peered":"2024-09-21T14:23:02.315446+0000","last_clean":"2024-09-21T14:23:02.315446+0000","last_became_active":"2024-09-21T14:22:35.215561+0000","last_became_peered":"2024-09-21T14:22:35.215561+0000","last_unstale":"2024-09-21T14:23:02.315446+0000","last_undegraded":"2024-09-21T14:23:02.315446+0000","last_fullsized":"2024-09-21T14:23:02.315446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:05.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058984800000000002,"stat_sum":{"num_bytes":27181375,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":6512,"num_read_kb":65589,"num_write":539,"num_write_kb":34390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":29327918,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.19","version":"115'777","reported_seq":5674,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.313876+0000","last_change":"2024-09-21T14:23:02.193776+0000","last_active":"2024-09-21T14:23:02.313876+0000","last_peered":"2024-09-21T14:23:02.313876+0000","last_clean":"2024-09-21T14:23:02.313876+0000","last_became_active":"2024-09-21T14:22:35.217004+0000","last_became_peered":"2024-09-21T14:22:35.217004+0000","last_unstale":"2024-09-21T14:23:02.313876+0000","last_undegraded":"2024-09-21T14:23:02.313876+0000","last_fullsized":"2024-09-21T14:23:02.313876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:44.918657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013530339999999999,"stat_sum":{"num_bytes":27514025,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":7350,"num_read_kb":30193,"num_write":1050,"num_write_kb":11024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":7435469,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.18","version":"115'274","reported_seq":1928,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.315922+0000","last_change":"2024-09-21T14:23:02.199748+0000","last_active":"2024-09-21T14:23:02.315922+0000","last_peered":"2024-09-21T14:23:02.315922+0000","last_clean":"2024-09-21T14:23:02.315922+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:23:02.315922+0000","last_undegraded":"2024-09-21T14:23:02.315922+0000","last_fullsized":"2024-09-21T14:23:02.315922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059072300000000003,"stat_sum":{"num_bytes":13244526,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":1650,"num_read_kb":12534,"num_write":235,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.17","version":"115'752","reported_seq":3123,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.276331+0000","last_change":"2024-09-21T14:23:02.200635+0000","last_active":"2024-09-21T14:23:02.276331+0000","last_peered":"2024-09-21T14:23:02.276331+0000","last_clean":"2024-09-21T14:23:02.276331+0000","last_became_active":"2024-09-21T14:22:35.217012+0000","last_became_peered":"2024-09-21T14:22:35.217012+0000","last_unstale":"2024-09-21T14:23:02.276331+0000","last_undegraded":"2024-09-21T14:23:02.276331+0000","last_fullsized":"2024-09-21T14:23:02.276331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:02.683415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744870000000001,"stat_sum":{"num_bytes":11195071,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4771,"num_read_kb":10236,"num_write":940,"num_write_kb":13984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":5304982,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.16","version":"113'488","reported_seq":2649,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193148+0000","last_change":"2024-09-21T14:23:02.193312+0000","last_active":"2024-09-21T14:23:02.193148+0000","last_peered":"2024-09-21T14:23:02.193148+0000","last_clean":"2024-09-21T14:23:02.193148+0000","last_became_active":"2024-09-21T14:22:29.161682+0000","last_became_peered":"2024-09-21T14:22:29.161682+0000","last_unstale":"2024-09-21T14:23:02.193148+0000","last_undegraded":"2024-09-21T14:23:02.193148+0000","last_fullsized":"2024-09-21T14:23:02.193148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:15.765317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098179499999999993,"stat_sum":{"num_bytes":7956855,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":3237,"num_read_kb":13890,"num_write":637,"num_write_kb":7256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":3589748,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.15","version":"115'751","reported_seq":2740,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196555+0000","last_change":"2024-09-21T14:23:02.196555+0000","last_active":"2024-09-21T14:23:02.196555+0000","last_peered":"2024-09-21T14:23:02.196555+0000","last_clean":"2024-09-21T14:23:02.196555+0000","last_became_active":"2024-09-21T14:21:18.149636+0000","last_became_peered":"2024-09-21T14:21:18.149636+0000","last_unstale":"2024-09-21T14:23:02.196555+0000","last_undegraded":"2024-09-21T14:23:02.196555+0000","last_fullsized":"2024-09-21T14:23:02.196555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050380200000000003,"stat_sum":{"num_bytes":37470362,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":4828,"num_read_kb":24612,"num_write":1036,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.14","version":"115'439","reported_seq":3154,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196786+0000","last_change":"2024-09-21T14:23:02.196786+0000","last_active":"2024-09-21T14:23:02.196786+0000","last_peered":"2024-09-21T14:23:02.196786+0000","last_clean":"2024-09-21T14:23:02.196786+0000","last_became_active":"2024-09-21T14:22:29.164131+0000","last_became_peered":"2024-09-21T14:22:29.164131+0000","last_unstale":"2024-09-21T14:23:02.196786+0000","last_undegraded":"2024-09-21T14:23:02.196786+0000","last_fullsized":"2024-09-21T14:23:02.196786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:06.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054163799999999999,"stat_sum":{"num_bytes":18588570,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":27,"num_read":4846,"num_read_kb":16747,"num_write":471,"num_write_kb":9905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":6514761,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146990+0000","last_change":"2024-09-21T14:22:32.174490+0000","last_active":"2024-09-21T14:23:01.146990+0000","last_peered":"2024-09-21T14:23:01.146990+0000","last_clean":"2024-09-21T14:23:01.146990+0000","last_became_active":"2024-09-21T14:22:32.174175+0000","last_became_peered":"2024-09-21T14:22:32.174175+0000","last_unstale":"2024-09-21T14:23:01.146990+0000","last_undegraded":"2024-09-21T14:23:01.146990+0000","last_fullsized":"2024-09-21T14:23:01.146990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:54.846715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"113'421","reported_seq":2965,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187447+0000","last_change":"2024-09-21T14:23:02.187650+0000","last_active":"2024-09-21T14:23:02.187447+0000","last_peered":"2024-09-21T14:23:02.187447+0000","last_clean":"2024-09-21T14:23:02.187447+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:23:02.187447+0000","last_undegraded":"2024-09-21T14:23:02.187447+0000","last_fullsized":"2024-09-21T14:23:02.187447+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066630900000000002,"stat_sum":{"num_bytes":38840074,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":3504,"num_read_kb":12859,"num_write":395,"num_write_kb":5718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146884+0000","last_change":"2024-09-21T14:22:26.141620+0000","last_active":"2024-09-21T14:23:01.146884+0000","last_peered":"2024-09-21T14:23:01.146884+0000","last_clean":"2024-09-21T14:23:01.146884+0000","last_became_active":"2024-09-21T14:22:26.141314+0000","last_became_peered":"2024-09-21T14:22:26.141314+0000","last_unstale":"2024-09-21T14:23:01.146884+0000","last_undegraded":"2024-09-21T14:23:01.146884+0000","last_fullsized":"2024-09-21T14:23:01.146884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:24:21.823458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"115'479","reported_seq":3029,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187569+0000","last_change":"2024-09-21T14:23:02.187569+0000","last_active":"2024-09-21T14:23:02.187569+0000","last_peered":"2024-09-21T14:23:02.187569+0000","last_clean":"2024-09-21T14:23:02.187569+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:23:02.187569+0000","last_undegraded":"2024-09-21T14:23:02.187569+0000","last_fullsized":"2024-09-21T14:23:02.187569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000612033,"stat_sum":{"num_bytes":27183207,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":3877,"num_read_kb":18030,"num_write":592,"num_write_kb":5955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146714+0000","last_change":"2024-09-21T14:22:26.141948+0000","last_active":"2024-09-21T14:23:01.146714+0000","last_peered":"2024-09-21T14:23:01.146714+0000","last_clean":"2024-09-21T14:23:01.146714+0000","last_became_active":"2024-09-21T14:22:26.141693+0000","last_became_peered":"2024-09-21T14:22:26.141693+0000","last_unstale":"2024-09-21T14:23:01.146714+0000","last_undegraded":"2024-09-21T14:23:01.146714+0000","last_fullsized":"2024-09-21T14:23:01.146714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:29:48.520733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"115'375","reported_seq":3089,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193514+0000","last_change":"2024-09-21T14:23:02.193514+0000","last_active":"2024-09-21T14:23:02.193514+0000","last_peered":"2024-09-21T14:23:02.193514+0000","last_clean":"2024-09-21T14:23:02.193514+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:23:02.193514+0000","last_undegraded":"2024-09-21T14:23:02.193514+0000","last_fullsized":"2024-09-21T14:23:02.193514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766750000000001,"stat_sum":{"num_bytes":40851495,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":2834,"num_read_kb":36189,"num_write":346,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.146543+0000","last_change":"2024-09-21T14:21:11.011737+0000","last_active":"2024-09-21T14:23:01.146543+0000","last_peered":"2024-09-21T14:23:01.146543+0000","last_clean":"2024-09-21T14:23:01.146543+0000","last_became_active":"2024-09-21T14:21:08.535483+0000","last_became_peered":"2024-09-21T14:21:08.535483+0000","last_unstale":"2024-09-21T14:23:01.146543+0000","last_undegraded":"2024-09-21T14:23:01.146543+0000","last_fullsized":"2024-09-21T14:23:01.146543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.710566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024570400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"115'512","reported_seq":5406,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.193596+0000","last_change":"2024-09-21T14:23:02.193596+0000","last_active":"2024-09-21T14:23:02.193596+0000","last_peered":"2024-09-21T14:23:02.193596+0000","last_clean":"2024-09-21T14:23:02.193596+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:23:02.193596+0000","last_undegraded":"2024-09-21T14:23:02.193596+0000","last_fullsized":"2024-09-21T14:23:02.193596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001210785,"stat_sum":{"num_bytes":42346199,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":6590,"num_read_kb":31308,"num_write":503,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1","version":"114'304","reported_seq":2558,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187730+0000","last_change":"2024-09-21T14:23:02.187953+0000","last_active":"2024-09-21T14:23:02.187730+0000","last_peered":"2024-09-21T14:23:02.187730+0000","last_clean":"2024-09-21T14:23:02.187730+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:23:02.187730+0000","last_undegraded":"2024-09-21T14:23:02.187730+0000","last_fullsized":"2024-09-21T14:23:02.187730+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095413400000000004,"stat_sum":{"num_bytes":36636910,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":3088,"num_read_kb":21572,"num_write":270,"num_write_kb":9367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"2.0","version":"17'117","reported_seq":216,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.144044+0000","last_change":"2024-09-21T14:22:29.157860+0000","last_active":"2024-09-21T14:23:01.144044+0000","last_peered":"2024-09-21T14:23:01.144044+0000","last_clean":"2024-09-21T14:23:01.144044+0000","last_became_active":"2024-09-21T14:22:29.157461+0000","last_became_peered":"2024-09-21T14:22:29.157461+0000","last_unstale":"2024-09-21T14:23:01.144044+0000","last_undegraded":"2024-09-21T14:23:01.144044+0000","last_fullsized":"2024-09-21T14:23:01.144044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:27.251019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":127,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.151401+0000","last_change":"2024-09-21T14:22:32.181750+0000","last_active":"2024-09-21T14:23:01.151401+0000","last_peered":"2024-09-21T14:23:01.151401+0000","last_clean":"2024-09-21T14:23:01.151401+0000","last_became_active":"2024-09-21T14:22:32.177114+0000","last_became_peered":"2024-09-21T14:22:32.177114+0000","last_unstale":"2024-09-21T14:23:01.151401+0000","last_undegraded":"2024-09-21T14:23:01.151401+0000","last_fullsized":"2024-09-21T14:23:01.151401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:00.138436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"115'747","reported_seq":3273,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.269616+0000","last_change":"2024-09-21T14:23:02.199483+0000","last_active":"2024-09-21T14:23:02.269616+0000","last_peered":"2024-09-21T14:23:02.269616+0000","last_clean":"2024-09-21T14:23:02.269616+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:23:02.269616+0000","last_undegraded":"2024-09-21T14:23:02.269616+0000","last_fullsized":"2024-09-21T14:23:02.269616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037525299999999998,"stat_sum":{"num_bytes":24563309,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":5340,"num_read_kb":124688,"num_write":970,"num_write_kb":60251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.2","version":"17'2","reported_seq":116,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.155641+0000","last_change":"2024-09-21T14:21:10.673500+0000","last_active":"2024-09-21T14:23:01.155641+0000","last_peered":"2024-09-21T14:23:01.155641+0000","last_clean":"2024-09-21T14:23:01.155641+0000","last_became_active":"2024-09-21T14:21:08.535051+0000","last_became_peered":"2024-09-21T14:21:08.535051+0000","last_unstale":"2024-09-21T14:23:01.155641+0000","last_undegraded":"2024-09-21T14:23:01.155641+0000","last_fullsized":"2024-09-21T14:23:01.155641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:56.898383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"115'1069","reported_seq":2489,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.286134+0000","last_change":"2024-09-21T14:23:02.187345+0000","last_active":"2024-09-21T14:23:02.286134+0000","last_peered":"2024-09-21T14:23:02.286134+0000","last_clean":"2024-09-21T14:23:02.286134+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:23:02.286134+0000","last_undegraded":"2024-09-21T14:23:02.286134+0000","last_fullsized":"2024-09-21T14:23:02.286134+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1069,"log_dups_size":0,"ondisk_log_size":1069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000411691,"stat_sum":{"num_bytes":17741403,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":4577,"num_read_kb":16301,"num_write":1532,"num_write_kb":8571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.143777+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:23:01.143777+0000","last_peered":"2024-09-21T14:23:01.143777+0000","last_clean":"2024-09-21T14:23:01.143777+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:23:01.143777+0000","last_undegraded":"2024-09-21T14:23:01.143777+0000","last_fullsized":"2024-09-21T14:23:01.143777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"114'533","reported_seq":2544,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192529+0000","last_change":"2024-09-21T14:23:02.192692+0000","last_active":"2024-09-21T14:23:02.192529+0000","last_peered":"2024-09-21T14:23:02.192529+0000","last_clean":"2024-09-21T14:23:02.192529+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:23:02.192529+0000","last_undegraded":"2024-09-21T14:23:02.192529+0000","last_fullsized":"2024-09-21T14:23:02.192529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053585799999999995,"stat_sum":{"num_bytes":62534416,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":3545,"num_read_kb":12364,"num_write":654,"num_write_kb":5627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T14:23:01.143953+0000","last_change":"2024-09-21T14:22:32.172034+0000","last_active":"2024-09-21T14:23:01.143953+0000","last_peered":"2024-09-21T14:23:01.143953+0000","last_clean":"2024-09-21T14:23:01.143953+0000","last_became_active":"2024-09-21T14:22:32.171774+0000","last_became_peered":"2024-09-21T14:22:32.171774+0000","last_unstale":"2024-09-21T14:23:01.143953+0000","last_undegraded":"2024-09-21T14:23:01.143953+0000","last_fullsized":"2024-09-21T14:23:01.143953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:41.624554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"113'593","reported_seq":3342,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192495+0000","last_change":"2024-09-21T14:23:02.192495+0000","last_active":"2024-09-21T14:23:02.192495+0000","last_peered":"2024-09-21T14:23:02.192495+0000","last_clean":"2024-09-21T14:23:02.192495+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:23:02.192495+0000","last_undegraded":"2024-09-21T14:23:02.192495+0000","last_fullsized":"2024-09-21T14:23:02.192495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043803100000000002,"stat_sum":{"num_bytes":53764636,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":5407,"num_read_kb":15634,"num_write":664,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.9","version":"115'610","reported_seq":4329,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.290206+0000","last_change":"2024-09-21T14:23:02.192888+0000","last_active":"2024-09-21T14:23:02.290206+0000","last_peered":"2024-09-21T14:23:02.290206+0000","last_clean":"2024-09-21T14:23:02.290206+0000","last_became_active":"2024-09-21T14:22:29.166889+0000","last_became_peered":"2024-09-21T14:22:29.166889+0000","last_unstale":"2024-09-21T14:23:02.290206+0000","last_undegraded":"2024-09-21T14:23:02.290206+0000","last_fullsized":"2024-09-21T14:23:02.290206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:48.958859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066009,"stat_sum":{"num_bytes":70625345,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":7151,"num_read_kb":84194,"num_write":658,"num_write_kb":55675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":51428954,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.a","version":"115'416","reported_seq":1815,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.202462+0000","last_change":"2024-09-21T14:23:02.202462+0000","last_active":"2024-09-21T14:23:02.202462+0000","last_peered":"2024-09-21T14:23:02.202462+0000","last_clean":"2024-09-21T14:23:02.202462+0000","last_became_active":"2024-09-21T14:22:35.217329+0000","last_became_peered":"2024-09-21T14:22:35.217329+0000","last_unstale":"2024-09-21T14:23:02.202462+0000","last_undegraded":"2024-09-21T14:23:02.202462+0000","last_fullsized":"2024-09-21T14:23:02.202462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:17:58.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027907600000000001,"stat_sum":{"num_bytes":13318421,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":21,"num_read":1268,"num_read_kb":8559,"num_write":404,"num_write_kb":7229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":11062623,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.b","version":"111'436","reported_seq":2332,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.202340+0000","last_change":"2024-09-21T14:23:02.202487+0000","last_active":"2024-09-21T14:23:02.202340+0000","last_peered":"2024-09-21T14:23:02.202340+0000","last_clean":"2024-09-21T14:23:02.202340+0000","last_became_active":"2024-09-21T14:21:18.144480+0000","last_became_peered":"2024-09-21T14:21:18.144480+0000","last_unstale":"2024-09-21T14:23:02.202340+0000","last_undegraded":"2024-09-21T14:23:02.202340+0000","last_fullsized":"2024-09-21T14:23:02.202340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:05.482086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026751500000000002,"stat_sum":{"num_bytes":33366655,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":2687,"num_read_kb":10392,"num_write":542,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.c","version":"114'664","reported_seq":2231,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.196243+0000","last_change":"2024-09-21T14:23:02.196567+0000","last_active":"2024-09-21T14:23:02.196243+0000","last_peered":"2024-09-21T14:23:02.196243+0000","last_clean":"2024-09-21T14:23:02.196243+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:23:02.196243+0000","last_undegraded":"2024-09-21T14:23:02.196243+0000","last_fullsized":"2024-09-21T14:23:02.196243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059781400000000001,"stat_sum":{"num_bytes":25328278,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3658,"num_read_kb":17839,"num_write":859,"num_write_kb":10999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.d","version":"113'506","reported_seq":2720,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195490+0000","last_change":"2024-09-21T14:23:02.196205+0000","last_active":"2024-09-21T14:23:02.195490+0000","last_peered":"2024-09-21T14:23:02.195490+0000","last_clean":"2024-09-21T14:23:02.195490+0000","last_became_active":"2024-09-21T14:22:36.212777+0000","last_became_peered":"2024-09-21T14:22:36.212777+0000","last_unstale":"2024-09-21T14:23:02.195490+0000","last_undegraded":"2024-09-21T14:23:02.195490+0000","last_fullsized":"2024-09-21T14:23:02.195490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:54:27.444035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079686199999999996,"stat_sum":{"num_bytes":18872961,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":3373,"num_read_kb":14313,"num_write":668,"num_write_kb":16264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5387372,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.e","version":"110'940","reported_seq":3644,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.192921+0000","last_change":"2024-09-21T14:23:02.193127+0000","last_active":"2024-09-21T14:23:02.192921+0000","last_peered":"2024-09-21T14:23:02.192921+0000","last_clean":"2024-09-21T14:23:02.192921+0000","last_became_active":"2024-09-21T14:22:23.539499+0000","last_became_peered":"2024-09-21T14:22:23.539499+0000","last_unstale":"2024-09-21T14:23:02.192921+0000","last_undegraded":"2024-09-21T14:23:02.192921+0000","last_fullsized":"2024-09-21T14:23:02.192921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:00:47.787223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089003300000000004,"stat_sum":{"num_bytes":58911077,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":6047,"num_read_kb":20604,"num_write":1393,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":7784436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.f","version":"115'450","reported_seq":4833,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195790+0000","last_change":"2024-09-21T14:23:02.195790+0000","last_active":"2024-09-21T14:23:02.195790+0000","last_peered":"2024-09-21T14:23:02.195790+0000","last_clean":"2024-09-21T14:23:02.195790+0000","last_became_active":"2024-09-21T14:22:36.216864+0000","last_became_peered":"2024-09-21T14:22:36.216864+0000","last_unstale":"2024-09-21T14:23:02.195790+0000","last_undegraded":"2024-09-21T14:23:02.195790+0000","last_fullsized":"2024-09-21T14:23:02.195790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:43.577416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036727999999999999,"stat_sum":{"num_bytes":63492156,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":5787,"num_read_kb":25840,"num_write":438,"num_write_kb":21462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":6327001,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.10","version":"115'378","reported_seq":2974,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.269389+0000","last_change":"2024-09-21T14:23:02.199468+0000","last_active":"2024-09-21T14:23:02.269389+0000","last_peered":"2024-09-21T14:23:02.269389+0000","last_clean":"2024-09-21T14:23:02.269389+0000","last_became_active":"2024-09-21T14:22:35.208525+0000","last_became_peered":"2024-09-21T14:22:35.208525+0000","last_unstale":"2024-09-21T14:23:02.269389+0000","last_undegraded":"2024-09-21T14:23:02.269389+0000","last_fullsized":"2024-09-21T14:23:02.269389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.874702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017700000000001,"stat_sum":{"num_bytes":36630188,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":2775,"num_read_kb":16167,"num_write":345,"num_write_kb":7253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":5015729,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.11","version":"111'331","reported_seq":2516,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.187780+0000","last_change":"2024-09-21T14:23:02.188002+0000","last_active":"2024-09-21T14:23:02.187780+0000","last_peered":"2024-09-21T14:23:02.187780+0000","last_clean":"2024-09-21T14:23:02.187780+0000","last_became_active":"2024-09-21T14:22:35.208396+0000","last_became_peered":"2024-09-21T14:22:35.208396+0000","last_unstale":"2024-09-21T14:23:02.187780+0000","last_undegraded":"2024-09-21T14:23:02.187780+0000","last_fullsized":"2024-09-21T14:23:02.187780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:35.143804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096473299999999995,"stat_sum":{"num_bytes":55838171,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":1890,"num_read_kb":23055,"num_write":386,"num_write_kb":18874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":22976540,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.12","version":"114'481","reported_seq":3147,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.301062+0000","last_change":"2024-09-21T14:23:02.187351+0000","last_active":"2024-09-21T14:23:02.301062+0000","last_peered":"2024-09-21T14:23:02.301062+0000","last_clean":"2024-09-21T14:23:02.301062+0000","last_became_active":"2024-09-21T14:22:29.157244+0000","last_became_peered":"2024-09-21T14:22:29.157244+0000","last_unstale":"2024-09-21T14:23:02.301062+0000","last_undegraded":"2024-09-21T14:23:02.301062+0000","last_fullsized":"2024-09-21T14:23:02.301062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:24.971948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042565000000000002,"stat_sum":{"num_bytes":26411215,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":4521,"num_read_kb":22673,"num_write":525,"num_write_kb":10769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5431126,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.13","version":"114'984","reported_seq":4285,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T14:23:02.195855+0000","last_change":"2024-09-21T14:23:02.196153+0000","last_active":"2024-09-21T14:23:02.195855+0000","last_peered":"2024-09-21T14:23:02.195855+0000","last_clean":"2024-09-21T14:23:02.195855+0000","last_became_active":"2024-09-21T14:22:41.248710+0000","last_became_peered":"2024-09-21T14:22:41.248710+0000","last_unstale":"2024-09-21T14:23:02.195855+0000","last_undegraded":"2024-09-21T14:23:02.195855+0000","last_fullsized":"2024-09-21T14:23:02.195855+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":984,"log_dups_size":0,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:41.665719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040070399999999999,"stat_sum":{"num_bytes":24156690,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":7083,"num_read_kb":24352,"num_write":1381,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":13233521,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1072465214,"num_objects":1379,"num_object_clones":324,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":624,"num_read":140284,"num_read_kb":947381,"num_write":20401,"num_write_kb":503383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2823,"num_bytes_recovered":268867858,"num_keys_recovered":619,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57163776,"data_stored":531111414,"data_compressed":30607838,"data_compressed_allocated":36036608,"data_compressed_original":513813665,"omap_allocated":0,"internal_metadata":0},"log_size":17218,"ondisk_log_size":17218,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98865,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55540,"kb_used_data":6376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94316300,"statfs":{"total":96636764160,"available":96579891200,"internally_reserved":0,"allocated":6529024,"data_stored":41922665,"data_compressed":2249393,"data_compressed_allocated":2801664,"data_compressed_original":39478747,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.25600000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.441},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34332,"kb_used_data":7308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337508,"statfs":{"total":96636764160,"available":96601608192,"internally_reserved":0,"allocated":7483392,"data_stored":55841343,"data_compressed":3409624,"data_compressed_allocated":3903488,"data_compressed_original":53456152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.41199999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.437}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.33200000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.27900000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.31}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56188,"kb_used_data":7016,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315652,"statfs":{"total":96636764160,"available":96579227648,"internally_reserved":0,"allocated":7184384,"data_stored":49301935,"data_compressed":2889364,"data_compressed_allocated":3411968,"data_compressed_original":46672523,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.7430000000000003,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.39000000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.32600000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.378}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61436,"kb_used_data":8144,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94310404,"statfs":{"total":96636764160,"available":96573853696,"internally_reserved":0,"allocated":8339456,"data_stored":58491366,"data_compressed":3384302,"data_compressed_allocated":3784704,"data_compressed_original":55290196,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.32000000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.38200000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.35099999999999998}]}]},{"osd":3,"up_from":0,"seq":55834574874,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574873,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":0,"seq":55834574874,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82904,"kb_used_data":7456,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94288936,"statfs":{"total":96636764160,"available":96551870464,"internally_reserved":0,"allocated":7634944,"data_stored":57913734,"data_compressed":3296429,"data_compressed_allocated":3616768,"data_compressed_original":55110634,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.246}]},{"osd":2,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.28799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.29999999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.44},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.38100000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6127616,"data_stored":56100424,"data_compressed":3216946,"data_compressed_allocated":3506176,"data_compressed_original":53963754,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12337152,"data_stored":119058163,"data_compressed":6888762,"data_compressed_allocated":8253440,"data_compressed_original":115661228,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":140569390,"data_compressed":7948729,"data_compressed_allocated":9515008,"data_compressed_original":137234462,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6778880,"data_stored":56650639,"data_compressed":3306180,"data_compressed_allocated":3678208,"data_compressed_original":54171988,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5750784,"data_stored":48055388,"data_compressed":2856954,"data_compressed_allocated":3366912,"data_compressed_original":46099083,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5763072,"data_stored":53351403,"data_compressed":3253300,"data_compressed_allocated":3706880,"data_compressed_original":51735832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5144576,"data_stored":41251292,"data_compressed":2248801,"data_compressed_allocated":2797568,"data_compressed_original":39466459,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:23:04.901 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:23:04.902 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:04.902 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T14:23:04.902 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:23:04.902 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:23:04.902 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T14:23:04.909 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:04.914 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:04.914 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:04.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:04.925 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:04.940 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:04.940+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:04.942 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:04.942 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:04.958 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:04.959 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:04.975 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:04.975 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:05.014 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:05.014 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:23:05.014 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.024922549000003, 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:05.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:05.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:05.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:05.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.528302729, 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:05.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.528277159000002, 2024-09-21T14:23:05.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:05.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:05.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.041 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.527322771000001, 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:05.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.52700827, 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:05.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.050 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.525593015999998, 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:05.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:05.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.055 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.524545566, 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:05.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.059 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.523262152000001, 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:05.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.521940925999999, 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.064 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.068 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.520899992, 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:05.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:05.072 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.520648552000001, 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.073 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:05.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.519602803000002, 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:05.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:05.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:05.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.519215860999999, 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:05.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.518280276999999, 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:05.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:05.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.518187608000002, 2024-09-21T14:23:05.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:05.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.094 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.514106389999998, 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:05.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:05.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.098 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.513923557999998, 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.513599845999998, 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:05.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:05.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.513478393, 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.107 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:05.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:05.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:05.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 31.512219794, 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.115 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.116 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:05.116 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:05.116+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:05.211 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:23:05.217 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:05.217+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:05.270 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.270 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:05.270 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:05.270 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.279 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:05.323 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:05.323+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:05.387 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:05.387 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:05.387 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:05.387 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.397 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:05.439 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:05.438+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:05.519 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:05.518+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109660474999998, 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.542 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:05.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:05.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:05.546 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109635424999993, 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.550 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109617272999998, 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:05.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:05.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:05.554 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109594036000004, 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:05.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:05.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109572968999998, 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:05.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:05.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109556608999995, 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109526329000005, 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:05.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109511190999996, 2024-09-21T14:23:05.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:05.575 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109491594999994, 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:05.579 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109482818000004, 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.583 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109470457, 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:05.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:05.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109461039999999, 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.588 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:05.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:05.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109439848999997, 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:05.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:05.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:05.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.596 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109422741000003, 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.600 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109405797999997, 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:05.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:05.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109389418999996, 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109348058999998, 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:05.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:05.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:05.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.109328378000001, 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:05.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.617 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.704439930999996, 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:05.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 13.341669100000001, 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:05.623 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:05.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:05.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:05.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:05.629 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:05.631 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:05.630+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:05.731 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:05.730+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:05.774 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.774 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:05.774 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:05.774 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.782 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:05.866 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:05.866+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:05.925 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:05.925 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:05.925 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:05.925 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:05.925 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:05.933 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:05.934 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:05.933+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:06.034 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:06.034+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481213659000005, 2024-09-21T14:23:06.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:06.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:06.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481196144999998, 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.083 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:06.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.087 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481178890999999, 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.091 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481151041000004, 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.092 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:06.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:06.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481129437000007, 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481112439, 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.100 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:06.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481095418999999, 2024-09-21T14:23:06.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.108 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481077968999998, 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:06.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.112 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481060830999994, 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:06.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481037603999994, 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.117 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:06.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.481021330000004, 2024-09-21T14:23:06.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:06.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480996958000006, 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:06.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480974391000004, 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480957470000007, 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:06.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480940970999995, 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:06.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:06.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480919974000003, 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:06.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:06.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 96.480897916000004, 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:06.147 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:06.151 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 73.306827044000002, 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:06.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:06.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.164593920000002, 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.157 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:06.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 40.931883843999998, 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:06.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:06.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:06.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:06.166 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:06.167 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:06.166+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:06.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:06.268 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:06.268+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:06.281 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:06.282 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:06.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:06.375 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:06.375+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:06.433 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:06.434 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:06.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:06.484 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:06.484+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:06.570 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:06.570+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:06.586 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 38.434411517999997, 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091527812999999, 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091512055000003, 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.600 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091490716000003, 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091474908000002, 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091458633999999, 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.091439905999998, 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090470467000003, 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090447834999999, 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090431195999997, 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090415219999997, 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090398069000003, 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090381307000001, 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.090173882999999, 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.089828675, 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.243957111, 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.233417588000002, 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.228203211, 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.207491625999999, 2024-09-21T14:23:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.204418474000001, 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:06.690 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:06.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:06.692 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:06.692+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:06.792 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:06.792+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:06.835 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:06.835 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:06.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:06.893 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:06.893+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:06.946 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:06.946 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:06.954 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:06.994 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:06.994+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:07.095 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:07.094+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:07.101 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.880004830000004, 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:23:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388180367999993, 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388162664000006, 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:23:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388146993000007, 2024-09-21T14:23:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388131478999995, 2024-09-21T14:23:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:23:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388116412000002, 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:23:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388101216999999, 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:23:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388086111000007, 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:23:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:23:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388071001, 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:23:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388055636000004, 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:23:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388040176999993, 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:23:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388025143999997, 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:23:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.388009255, 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:23:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:23:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.387983578999993, 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:23:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:23:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.387968294000004, 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.387946330000005, 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.387930346999994, 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:23:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:23:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.337388959000002, 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:23:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.873335203000003, 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 38.940424219000001, 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:07.189 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:07.190 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:07.191 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:07.191 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:07.192 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:07.192 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:07.192 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:07.192 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:07.193 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:07.196 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:07.196 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:07.196+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:07.214 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:07.214 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:07.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:23:07.235 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:07.235 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:07.296 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:07.297+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:07.305 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.305 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:07.397 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:07.397+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:07.462 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.462 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:07.498 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:07.498+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:07.573 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.573 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:07.625 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:07.625+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:07.684 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.684 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:07.733 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:07.733+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:07.800 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:07.799+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:07.833 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.833 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:07.901 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:07.902+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:07.943 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:07.943 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:07.951 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:08.002 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:08.002+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:08.057 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.057 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.066 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:08.110 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:08.109+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:08.168 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.168 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:08.204 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:08.204+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:23:08.304 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:08.304+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:08.334 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.334 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.344 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:08.405 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:08.404+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:08.446 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.446 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.455 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:08.539 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:08.538+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:08.597 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.598 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:08.606 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:08.606+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:08.706 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:08.705+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:08.748 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.748 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:08.807 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:08.807+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:08.900 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:08.900 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:08.907 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:08.908+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:08.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:09.008 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:09.009+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:09.011 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:09.011 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:09.020 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:09.109 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:09.109+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:09.123 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:09.123 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:09.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:09.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:09.209+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:09.244 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:09.244 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:09.248 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:09.254 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:09.310 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:09.310+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:09.388 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:09.388 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:09.402 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:09.402 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:09.405 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:09.406 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:09.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:09.411 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:09.411+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:09.421 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:09.436 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:09.454 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:09.454 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:09.470 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:09.470 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:09.486 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:09.486 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:09.503 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:09.503 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:09.511 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:09.510+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920960518000001, 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.518 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.520 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920943003999994, 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.522 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920925749999995, 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.526 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:09.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:09.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:09.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.9208979, 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.530 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:09.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:09.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920876296000003, 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.534 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:09.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920859297999996, 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:09.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:09.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:09.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920842277999995, 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.542 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920824827999994, 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.546 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920807690000004, 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:09.554 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920784463000004, 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:09.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920768189, 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920743817000002, 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.92072125, 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920704329000003, 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920687830000006, 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.574 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920666832999999, 2024-09-21T14:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:09.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:09.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 99.920644775, 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.582 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:09.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:09.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:09.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 76.746573902999998, 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.586 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:09.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:09.591 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.604340778999997, 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:09.595 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.371630703000001, 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:09.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:09.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:09.599 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:09.600 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:09.642 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:09.642+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:09.703 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:09.703 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:09.703 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:09.703 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:09.712 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:09.713 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:09.713+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:09.813 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:09.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:09.813 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:09.813 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:09.813 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:09.814 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:09.813+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:09.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:09.914 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:09.914+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:09.966 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:09.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.814450032000003, 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:09.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:09.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:09.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.971 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84604 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471566326999998, 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053072978, 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84604 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.972 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494281+0000", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494280+0000", 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494283+0000", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494288+0000", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.755e-06 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494293+0000", 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4569999999999998e-06 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547183+0000", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052890118 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547238+0000", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5053000000000003e-05 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547354+0000", 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011582900000000001 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471550569000001, 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471529230000002, 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:09.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:09.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.983 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471513422000001, 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:09.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:09.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471497147999997, 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.471478419999997, 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:09.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:09.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470508981000002, 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:09.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:09.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:09.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:09.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470486348999998, 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.000 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:10.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:10.003 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470469710000003, 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.004 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:10.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.007 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470453734000003, 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.011 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470436583000001, 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.012 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:10.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470419821, 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.016 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:10.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:10.020 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.470212396999997, 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:10.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:10.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.469867188999999, 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.623995624999999, 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:10.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:10.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.613456102000001, 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.608241724999999, 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:10.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.587530139999998, 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.049 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.584456987999999, 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:10.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:10.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:10.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.062 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.062 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:10.062 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:10.063 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:23:10.063 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:23:10.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:10.064 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:10.064+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:10.164 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:10.164+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:10.209 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:10.210 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:10.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:23:10.264 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:10.264+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:10.336 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.336 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:10.336 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:10.336 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:10.336 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:10.390 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:10.390+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:10.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 123, 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "da87574b-7df0-407c-8805-347c49d2d055", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-21T14:20:56.341644+0000", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-21T14:23:10.224862+0000", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-21T14:21:06.519050+0000", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-21T14:22:31.158403+0000", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5806080, 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-21T14:23:10.430 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 4, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 3, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 8, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "reef", 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-21T14:23:10.431 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:07.255609+0000", 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:23:10.432 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.433 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 3, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 17, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:23:10.434 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:23:10.435 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.9800000190734863, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.9800000190734863, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:23:10.436 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:08.449794+0000", 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:23:10.437 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:23:10.438 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "17", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:23:10.439 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:23:10.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.441 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 8, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 8, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.25, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:23:10.442 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "pool_client.0", 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-21T14:21:16.638770+0000", 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 2, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 1, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 32, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 32, 2024-09-21T14:23:10.443 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 32, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 32, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 32, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "123", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-21T14:23:10.444 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 75, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 123, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-21T14:23:10.445 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:23:10.446 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 2.2400000095367432, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 2.2400000095367432, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.62999999523162842, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.96931648254394531, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T14:23:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "4cd103e2-3667-49ee-9ab4-f3b9e9a18d18", 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 104, 2024-09-21T14:23:10.448 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.450 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3182191515 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6801/3182191515", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6803/3182191515", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/3182191515", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6805/3182191515", 2024-09-21T14:23:10.452 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b7cc4ae1-1649-4592-91e7-4814ad9aa260", 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 82, 2024-09-21T14:23:10.453 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.454 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.455 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3526445386 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6809/3526445386", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6811/3526445386", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/3526445386", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6813/3526445386", 2024-09-21T14:23:10.457 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "adda4736-6878-49a4-ae7c-77b97c866703", 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 19, 2024-09-21T14:23:10.458 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6816", 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6817", 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.459 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6818", 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6819", 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.460 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6822", 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6823", 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6820", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6821", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3383683073 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6817/3383683073", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6819/3383683073", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6823/3383683073", 2024-09-21T14:23:10.462 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6821/3383683073", 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "af765f77-37b5-4efd-89f0-4dfe5ef0978f", 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 0, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.7545318603515625, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.463 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 50, 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6824", 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6825", 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.464 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6826", 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6827", 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.465 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6830", 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6831", 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.466 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6828", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6829", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3204248728 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6825/3204248728", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6827/3204248728", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6831/3204248728", 2024-09-21T14:23:10.467 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6829/3204248728", 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c", 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.468 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6800", 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6801", 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.469 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6802", 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6803", 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.470 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6806", 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6807", 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.471 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6804", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6805", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3444590304 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6801/3444590304", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6803/3444590304", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6807/3444590304", 2024-09-21T14:23:10.472 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6805/3444590304", 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "5daf6d5d-18e7-475d-87ae-4b15965a8192", 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.473 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6808", 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6809", 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6810", 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6811", 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6814", 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6815", 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6812", 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6813", 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4067730938 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6809/4067730938", 2024-09-21T14:23:10.477 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6811/4067730938", 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6815/4067730938", 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6813/4067730938", 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e75e58a4-a37f-4029-b4fb-7ba22db2d6f0", 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.478 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 105, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6816", 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.479 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6817", 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6818", 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6819", 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.480 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6822", 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6823", 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6820", 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6821", 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2060683966 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6817/2060683966", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6819/2060683966", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6823/2060683966", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6821/2060683966", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "c0378ee4-c4f7-4430-ac1e-5740e2e8bfae", 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-21T14:23:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 13, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 88, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6824", 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.484 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6825", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6826", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6827", 2024-09-21T14:23:10.485 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6830", 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6831", 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6828", 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.161:6829", 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2285860342 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.161:6825/2285860342", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.161:6827/2285860342", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.161:6831/2285860342", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.161:6829/2285860342", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.488 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.729613+0000", 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.843490+0000", 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.489 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.767547+0000", 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.490 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 65536, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.797830+0000", 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.760985+0000", 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.491 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.830324+0000", 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.747245+0000", 2024-09-21T14:23:10.492 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540138322906710015, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-21T14:21:03.849223+0000", 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.493 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "1.1", 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.3", 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: 5 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.d", 2024-09-21T14:23:10.494 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: 2 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.e", 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: 7 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [ 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.495 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "3.13", 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "mappings": [ 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "from": 4, 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "to": 5 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [], 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-21T14:23:10.496 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [ 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "snaps": [ 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "begin": 26, 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: "length": 8 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.497 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [ 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "snaps": [ 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "begin": 26, 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "length": 1 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-21T14:23:10.498 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.499 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.501 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': 'da87574b-7df0-407c-8805-347c49d2d055', 'created': '2024-09-21T14:20:56.341644+0000', 'modified': '2024-09-21T14:23:10.224862+0000', 'last_up_change': '2024-09-21T14:21:06.519050+0000', 'last_in_change': '2024-09-21T14:22:31.158403+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:21:07.255609+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.9800000190734863, 'score_stable': 2.9800000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:21:08.449794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:21:16.638770+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 75, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.9693164825439453, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '4cd103e2-3667-49ee-9ab4-f3b9e9a18d18', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6801', 'nonce': 3182191515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6803', 'nonce': 3182191515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6807', 'nonce': 3182191515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 3182191515}, {'type': 'v1', 'addr': '172.21.15.73:6805', 'nonce': 3182191515}]}, 'public_addr': '172.21.15.73:6801/3182191515', 'cluster_addr': '172.21.15.73:6803/3182191515', 'heartbeat_back_addr': '172.21.15.73:6807/3182191515', 'heartbeat_front_addr': '172.21.15.73:6805/3182191515', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b7cc4ae1-1649-4592-91e7-4814ad9aa260', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6809', 'nonce': 3526445386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6811', 'nonce': 3526445386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6815', 'nonce': 3526445386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 3526445386}, {'type': 'v1', 'addr': '172.21.15.73:6813', 'nonce': 3526445386}]}, 'public_addr': '172.21.15.73:6809/3526445386', 'cluster_addr': '172.21.15.73:6811/3526445386', 'heartbeat_back_addr': '172.21.15.73:6815/3526445386', 'heartbeat_front_addr': '172.21.15.73:6813/3526445386', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'adda4736-6878-49a4-ae7c-77b97c866703', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6816', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6817', 'nonce': 3383683073}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6818', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6819', 'nonce': 3383683073}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6822', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6823', 'nonce': 3383683073}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6820', 'nonce': 3383683073}, {'type': 'v1', 'addr': '172.21.15.73:6821', 'nonce': 3383683073}]}, 'public_addr': '172.21.15.73:6817/3383683073', 'cluster_addr': '172.21.15.73:6819/3383683073', 'heartbeat_back_addr': '172.21.15.73:6823/3383683073', 'heartbeat_front_addr': '172.21.15.73:6821/3383683073', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'af765f77-37b5-4efd-89f0-4dfe5ef0978f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.7545318603515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6824', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6825', 'nonce': 3204248728}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6826', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6827', 'nonce': 3204248728}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6830', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6831', 'nonce': 3204248728}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6828', 'nonce': 3204248728}, {'type': 'v1', 'addr': '172.21.15.73:6829', 'nonce': 3204248728}]}, 'public_addr': '172.21.15.73:6825/3204248728', 'cluster_addr': '172.21.15.73:6827/3204248728', 'heartbeat_back_addr': '172.21.15.73:6831/3204248728', 'heartbeat_front_addr': '172.21.15.73:6829/3204248728', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6800', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6801', 'nonce': 3444590304}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6802', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6803', 'nonce': 3444590304}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6806', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6807', 'nonce': 3444590304}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6804', 'nonce': 3444590304}, {'type': 'v1', 'addr': '172.21.15.161:6805', 'nonce': 3444590304}]}, 'public_addr': '172.21.15.161:6801/3444590304', 'cluster_addr': '172.21.15.161:6803/3444590304', 'heartbeat_back_addr': '172.21.15.161:6807/3444590304', 'heartbeat_front_addr': '172.21.15.161:6805/3444590304', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5daf6d5d-18e7-475d-87ae-4b15965a8192', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6808', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6809', 'nonce': 4067730938}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6810', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6811', 'nonce': 4067730938}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6814', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6815', 'nonce': 4067730938}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6812', 'nonce': 4067730938}, {'type': 'v1', 'addr': '172.21.15.161:6813', 'nonce': 4067730938}]}, 'public_addr': '172.21.15.161:6809/4067730938', 'cluster_addr': '172.21.15.161:6811/4067730938', 'heartbeat_back_addr': '172.21.15.161:6815/4067730938', 'heartbeat_front_addr': '172.21.15.161:6813/4067730938', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e75e58a4-a37f-4029-b4fb-7ba22db2d6f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6816', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6817', 'nonce': 2060683966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6818', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6819', 'nonce': 2060683966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6822', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6823', 'nonce': 2060683966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6820', 'nonce': 2060683966}, {'type': 'v1', 'addr': '172.21.15.161:6821', 'nonce': 2060683966}]}, 'public_addr': '172.21.15.161:6817/2060683966', 'cluster_addr': '172.21.15.161:6819/2060683966', 'heartbeat_back_addr': '172.21.15.161:6823/2060683966', 'heartbeat_front_addr': '172.21.15.161:6821/2060683966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c0378ee4-c4f7-4430-ac1e-5740e2e8bfae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6824', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6825', 'nonce': 2285860342}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6826', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6827', 'nonce': 2285860342}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6830', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6831', 'nonce': 2285860342}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6828', 'nonce': 2285860342}, {'type': 'v1', 'addr': '172.21.15.161:6829', 'nonce': 2285860342}]}, 'public_addr': '172.21.15.161:6825/2285860342', 'cluster_addr': '172.21.15.161:6827/2285860342', 'heartbeat_back_addr': '172.21.15.161:6831/2285860342', 'heartbeat_front_addr': '172.21.15.161:6829/2285860342', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.729613+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.843490+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.767547+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.797830+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.760985+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.830324+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.747245+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:21:03.849223+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.1', 'osds': [5, 4]}, {'pgid': '3.3', 'osds': [6, 5]}, {'pgid': '3.d', 'osds': [6, 2]}, {'pgid': '3.e', 'osds': [4, 7]}], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 4, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 8}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:23:10.501 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.e 2024-09-21T14:23:10.501 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.e 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.234279225999998, 2024-09-21T14:23:10.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:23:10.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:23:10.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:23:10.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742454764000001, 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.507 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:23:10.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.74243706, 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742421389, 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.515 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742405875000003, 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.519 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742390807999996, 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.523 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:23:10.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:23:10.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742375612999993, 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:23:10.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742360507000001, 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742345396999994, 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:23:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:23:10.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.538 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742330031999998, 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:10.542 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742314573000002, 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:23:10.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742299540000005, 2024-09-21T14:23:10.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:10.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:23:10.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742283650999994, 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.550 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742257975000001, 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:23:10.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742242689999998, 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:23:10.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.561 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742220725999999, 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:23:10.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.565 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.742204743000002, 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.691663355000003, 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:23:10.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 45.227609598999997, 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:23:10.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:10.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 42.294698615000001, 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:23:10.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:10.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:23:10.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:23:10.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.584 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:10.585 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:10.500+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:10.628 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:10.628+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:10.690 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.690 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:10.690 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:10.690 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:10.702 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:10.701+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:10.802 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:10.802+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:10.846 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.846 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:10.846 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:10.846 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:10.846 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:10.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:10.903 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:10.903+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:10.958 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:10.958 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:10.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:10.958 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:10.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.565899335, 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:10.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:10.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.962 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.565888434, 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:10.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:10.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:10.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.966 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.56587802600001, 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.565870085, 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:10.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:10.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48913210000001, 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48911369, 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:10.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:10.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:10.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.982 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48908794099999, 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.489066912, 2024-09-21T14:23:10.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.489041062, 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48902258299999, 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:10.994 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:10.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:10.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:10.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48900446099999, 2024-09-21T14:23:10.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:10.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:11.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48898409900001, 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.002 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:11.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:11.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:11.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.488964407, 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.006 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:11.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:11.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48894742900001, 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.488927757, 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.014 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48889829399999, 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.018 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.488856048, 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:11.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.48883705999999, 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:11.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:11.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.029 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.488813702, 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.488790349, 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:11.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.037 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:11.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:11.039 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:11.039+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:11.139 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:11.140+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:11.141 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:11.141 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:11.141 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:11.141 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:11.145 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:23:11.147 INFO:teuthology.orchestra.run.smithi073.stderr:clear 3.e pg_upmap mapping 2024-09-21T14:23:11.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:11.241 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:11.241+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:11.269 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:11.269 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:11.270 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:11.270 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:11.270 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:11.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:11.363 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:11.363+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 121.88069094799999, 2024-09-21T14:23:11.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:11.429 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035114586000006, 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035100553999996, 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:11.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:11.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035084554999997, 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.437 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:11.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:11.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035069659000001, 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.441 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035054723000002, 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.445 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:11.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035039742999999, 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:11.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035025126999997, 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.035010080000006, 2024-09-21T14:23:11.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034995520999999, 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:11.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:11.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034980795999999, 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:11.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:11.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034966355999998, 2024-09-21T14:23:11.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:11.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034952005999997, 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:11.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034931940000007, 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034917030000003, 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034902090000003, 2024-09-21T14:23:11.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:11.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.034882397000004, 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:11.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:11.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.489 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.387401508000004, 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:11.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:11.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:11.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:11.493 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.277252324999999, 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.498 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.373689781000003, 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.499 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:11.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:11.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:11.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:11.504 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:11.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:11.505 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:11.505+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:11.515 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:11.606 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:11.606+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:11.615 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:11.615 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:11.615 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:11.615 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:11.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:11.664 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:11.664 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:11.680 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:11.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:11.696 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:11.707 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:11.707+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:11.711 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:11.727 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:11.727 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:11.727 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:11.728 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:11.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:11.743 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:11.743 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:11.759 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:11.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:11.775 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:11.775 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:11.827 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:11.827+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:11.839 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:11.839 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:11.848 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:11.910 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:11.910+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:11.949 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:11.950 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:11.959 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:12.011 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:12.011+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:12.100 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.100 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:12.111 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:12.112+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:12.212 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:12.212+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:12.213 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.213 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.222 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:12.312 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:12.313+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:12.325 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.326 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.334 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:12.414 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:12.414+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:12.478 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.478 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:12.514 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:12.514+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:12.590 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.590 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:12.615 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:12.616+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:12.702 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.702 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:12.716 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:12.717+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:12.813 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.813 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.818 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:12.818+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:12.822 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:12.919 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:12.919+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:12.924 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:12.924 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:12.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:13.019 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:13.020+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:13.045 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.046 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.054 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:13.120 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:13.120+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:13.158 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.159 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:23:13.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:13.221 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:13.222+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:13.313 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.313 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:13.323 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:13.322+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:13.423 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:13.423+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:13.466 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.467 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:13.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:13.524+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:13.618 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.618 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.626 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:13.626+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:13.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:13.726 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:13.726+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:13.728 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.728 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:13.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:13.831 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:13.831+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:13.841 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:13.841 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:13.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:13.912 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:13.913 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:13.928 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:13.928+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:13.931 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:13.931 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:13.960 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:13.977 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:13.977 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:13.995 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:13.995 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:14.011 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:14.012 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:14.013 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:14.014 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:14.014 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:14.014 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:14.014 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:14.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:14.029 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:14.030 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:14.030 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:14.030+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:14.048 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:14.048 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:14.129 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:14.129+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.138 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.745422697, 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:14.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:14.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:14.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.745411796, 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.146 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.745401388, 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:14.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:14.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.745393447, 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:14.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668655462, 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66863705199999, 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.159 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:14.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:14.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:14.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66861130300001, 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:14.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668590274, 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.167 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:14.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668564424, 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.171 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66854594500001, 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66852782300001, 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:14.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:14.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:14.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668507461, 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668487769, 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668470791, 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:14.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668451119, 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:14.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:14.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:14.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66842165600001, 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.202 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66837941, 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:14.206 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.66836042200001, 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668337064, 2024-09-21T14:23:14.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:14.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.668313711, 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.214 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:14.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:14.230 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:14.230+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:14.242 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:23:14.331 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:14.331+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:14.350 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:14.350 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:14.350 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:14.350 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:14.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:14.432 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:14.432+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:14.462 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:14.462 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:14.462 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:14.462 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:14.462 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:14.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:14.556 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:14.556+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 125.074733393, 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:14.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:14.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:14.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229157031, 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:23:14.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:23:14.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229142999000004, 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.627 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229127000000005, 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229112103999995, 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229097167999996, 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229082188000007, 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229067572000005, 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.649 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229052525, 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229037966000007, 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229023240999993, 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.229008801000006, 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.228994451000005, 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.228974385000001, 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.228959474999996, 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:14.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:14.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:14.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.228944534999997, 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:14.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 87.228924841999998, 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.684 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.581443952999997, 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:14.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:14.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.47129477, 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:14.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:14.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 49.567732225999997, 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.694 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:14.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:14.699 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:14.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:14.701 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:14.700+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:14.801 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:14.802+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:14.802 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:14.803 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:14.811 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:14.902 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:14.902+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:14.913 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:14.913 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:14.914 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:14.914 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:14.914 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:14.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:15.003 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:15.004+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:15.026 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.026 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.259001931, 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:15.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:15.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.258925421000001, 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:15.031 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.258115457000002, 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:15.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:15.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:15.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.258088399999998, 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.040 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:15.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:15.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257640176999999, 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:15.044 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257617836999998, 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257576145999998, 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:15.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:15.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:15.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.057 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257417480999997, 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:15.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:15.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257385059999997, 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.062 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:15.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:15.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257356530999999, 2024-09-21T14:23:15.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:15.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257335435000002, 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257303264999997, 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:15.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:15.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257226776000003, 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:15.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:15.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:15.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:15.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.083 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257203836000002, 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:15.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:15.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257186818999998, 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.088 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.090 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:15.092 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257037965999999, 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257022255999999, 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.257006832999998, 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.101 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.256874768000003, 2024-09-21T14:23:15.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:15.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:15.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:15.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.256839710999998, 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.110 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.114 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.115 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:15.116 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:15.116+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:15.216 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:15.216+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:15.261 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.261 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:15.261 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:15.261 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.273 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:15.357 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:15.357+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:15.415 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.415 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:15.415 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:15.415 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:15.415 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.418 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:15.418+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:15.424 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:15.518 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:15.519+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:15.566 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 51.564203831, 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:15.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:15.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:15.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.067584011000001, 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.576 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.067558441000003, 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:15.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.066604052999999, 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.066289552000001, 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.588 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.064874297999999, 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:15.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.063826847999998, 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.593 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:15.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:15.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.062543433999998, 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.597 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:15.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.061222207999997, 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.601 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:15.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:15.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.060181274000001, 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:15.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:15.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:15.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.059929834000002, 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.609 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:15.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:15.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:15.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.058884085000003, 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.613 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:15.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.058497142999997, 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.617 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.057561559, 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:15.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.057468890000003, 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:15.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.053387671999999, 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.629 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.053204839999999, 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.633 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.052881128000003, 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:15.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:15.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.052759674999997, 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.641 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 42.051501076000001, 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:15.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:15.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.648 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.649 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:15.650 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:15.650+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:15.750 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:15.751+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:15.751 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.751 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:15.751 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:15.752 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:15.851 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:15.852+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:15.861 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.861 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:15.861 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:15.861 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:15.861 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:15.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:15.951 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:15.951+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539976743, 2024-09-21T14:23:15.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.976 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.53995169300001, 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:15.980 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539933541, 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539910304, 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:15.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:15.988 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539889237, 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.53987287699999, 2024-09-21T14:23:15.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:15.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:15.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:15.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539842597, 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:15.996 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:15.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:15.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:15.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.53982745899999, 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.000 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.53980786299999, 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.004 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:16.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:16.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539799086, 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.008 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:16.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539786725, 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.012 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.013 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539777308, 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.016 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.017 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:16.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539756117, 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:16.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:16.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:16.023 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539739009, 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.024 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:16.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:16.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539722066, 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:16.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:16.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:16.031 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.53970568699999, 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.035 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539664327, 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.038 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.039 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 106.539644646, 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.043 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.134756198999995, 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.044 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:16.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 23.771985367999999, 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:16.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:16.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:16.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:16.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:16.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:16.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:16.055 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:16.056 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:16.056 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:16.056+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:16.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:16.157 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:16.157+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:16.164 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:16.165 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:16.165 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T14:23:16.165 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2024-09-21T14:23:16.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0 2024-09-21T14:23:16.206 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:16.206 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:16.212 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:16.212 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:16.212 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:16.213 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:16.224 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:16.226 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:16.226 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:16.242 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:16.242 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:16.257 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:23:16.264 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:16.264 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:16.267 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:16.267+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:16.281 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:16.281 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:16.305 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:16.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:16.327 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:16.344 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:16.370 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:16.370 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:16.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:16.397 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:16.397+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:16.460 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:16.460+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:16.481 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:16.481 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:16.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:16.561 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:16.561+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:16.595 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:16.595 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:16.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:16.689 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:16.689+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:16.746 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:16.746 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:16.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:16.762 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:16.762+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:16.862 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:16.862+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:16.897 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:16.898 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:16.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:16.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:16.963+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.057 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:17.100 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:17.100+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:17.159 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.160 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.165 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:17.165+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:17.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:17.263 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.7 primary-affinity to 0 (802) 2024-09-21T14:23:17.263 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:23:17.266 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:17.266+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:17.319 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.319 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:17.367 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:17.367+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:17.468 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:17.468+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:17.473 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.473 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:17.568 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:17.568+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:17.624 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.624 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:17.676 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:17.676+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:17.733 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.733 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.741 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:17.782 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:17.782+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:17.870 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:17.870+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:17.884 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:17.884 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:17.892 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:17.977 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:17.977+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:18.036 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:18.036 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:18.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:18.072 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:18.072+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:18.172 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:18.173+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:18.188 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:18.188 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:18.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:18.274 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:18.275+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:18.325 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:18.325 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:18.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:18.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:18.380 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:18.380+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:18.438 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:18.447 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:18.476 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:18.489 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:18.489+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:18.492 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:18.492 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:18.508 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:18.508 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:18.523 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:18.523 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:18.539 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:18.556 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:18.556 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:18.571 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:18.571 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:18.577 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:18.577+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:18.587 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:18.593 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:18.593 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.825908210000001, 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.594 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:18.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.825831700000002, 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.602 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.825021735999997, 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.606 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824994679, 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.607 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:18.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:18.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824546456, 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.611 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824524115999999, 2024-09-21T14:23:18.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:18.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:18.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.619 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824482424999999, 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:18.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:18.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824323759999999, 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:18.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:18.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:18.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824291338999998, 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:18.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:18.632 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.82426281, 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.636 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824241714000003, 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824209543999999, 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824133054999997, 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:18.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824110115000003, 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:18.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:18.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.656 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.824093097999999, 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:18.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:18.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.823944245, 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.661 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.823928535, 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:18.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.823913112, 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:18.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.823781046999997, 2024-09-21T14:23:18.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.678 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.823745989999999, 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.679 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:18.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:18.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:18.683 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:18.685 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:18.685+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:18.685 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:18.785 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:18.786+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:18.788 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:18.788 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:18.788 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:18.788 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:18.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:18.886 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:18.887+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:18.939 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:18.939 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:18.939 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:18.939 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:18.939 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:18.947 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:18.987 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:18.988+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:19.088 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:19.089+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:19.090 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.088161919000001, 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:19.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:19.095 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.591542099000002, 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.096 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:19.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.099 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.591516529000003, 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.100 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.590562140999999, 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.590247640000001, 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:19.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:19.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.588832386, 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:19.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:19.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:19.115 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.587784935999998, 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.586501521999999, 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:19.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.123 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.585180295999997, 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:19.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.584139362000002, 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.583887922000002, 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:19.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.582842173000003, 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.582455230999997, 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:19.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:19.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:19.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.581519647, 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:19.147 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.581426978000003, 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:19.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.57734576, 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:19.151 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:19.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:19.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:19.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.577162928, 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.155 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.159 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.576839216000003, 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:19.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:19.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.163 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.576717762999998, 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:19.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:19.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:19.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 45.575459164000002, 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:19.171 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.172 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.172 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.172 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:19.216 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:19.217+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:19.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:23:19.281 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.281 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:19.281 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:19.281 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.290 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:19.290+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:19.291 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:19.391 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:19.391+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:19.441 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.441 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:19.441 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:19.441 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:19.442 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.450 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:19.494 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:19.495+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:19.553 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121638668, 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.554 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:19.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121613618, 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121595466, 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.12157222899999, 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121551162, 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121534802, 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121504522, 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121489384, 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121469788, 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.12146101099999, 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.12144865, 2024-09-21T14:23:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121439233, 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121418042, 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.12140093399999, 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:19.609 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121383991, 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:19.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121367612, 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.614 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:19.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.121326252, 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.618 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.12130657100001, 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:19.626 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 86.716418124, 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:19.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:19.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:19.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:19.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.631 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 27.353647293000002, 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:19.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:19.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:19.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:19.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:19.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:19.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:19.638 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:19.639 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:19.639+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:19.739 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:19.739+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.782 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:19.782 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.790 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:19.840 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:19.841+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:19.932 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:19.932 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:19.932 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:19.932 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:19.932 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:19.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:19.941 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:19.941+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:20.041 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:20.041+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:20.043 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:20.043 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44694520100001, 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:20.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.045 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.046 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446927687, 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.048 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.049 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.050 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.051 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.052 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446910433, 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:20.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.056 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446882583, 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:20.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:20.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44686097899999, 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:20.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:20.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.064 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446843981, 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:20.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:20.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.068 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446826961, 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:20.070 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.071 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:20.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.073 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.073 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:20.073 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446809511, 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.074 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.075 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.076 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:20.077 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44679237299999, 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.078 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:20.079 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:20.080 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:20.081 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44676914599999, 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:20.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:20.083 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.084 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.085 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446752872, 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.086 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.087 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:20.088 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:20.089 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44672850000001, 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:20.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:20.091 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:20.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.093 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446705933, 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.094 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.44668901199999, 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446672513, 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:20.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446651516, 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.109 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 110.446629458, 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.110 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.272558586000002, 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:20.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:20.119 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.130325462000002, 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:20.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:20.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 54.897615385999998, 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:20.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:20.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:20.127 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:20.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:20.173 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:20.173+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:20.238 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:20.238 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:20.243 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:20.244+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:20.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:20.275 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:23:20.344 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:20.344+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:20.385 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:20.385 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:20.385 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:20.385 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:20.385 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:20.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:20.445 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:20.445+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:20.499 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:20.499 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.347323250999999, 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.500 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:20.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.004423787999997, 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:20.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:20.509 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.004402448999997, 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:20.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:20.513 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.004386641000004, 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.517 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.004370367, 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:20.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:20.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:20.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.004351638999999, 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:20.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003382199999997, 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.526 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003359568, 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.530 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003342928999999, 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:20.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:20.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003326952999998, 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.538 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003309801999997, 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:20.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:20.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:20.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:20.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003293040000003, 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:20.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:20.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:20.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:20.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.003085616, 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:20.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:20.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:20.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.002740408000001, 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:20.554 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:20.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:20.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.156868844000002, 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:20.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.146329321000003, 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:20.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:20.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:20.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:20.571 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.141114944000002, 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:20.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:20.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:20.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:20.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.120403359000001, 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:20.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:20.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:20.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:20.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.117330207000002, 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:20.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:20.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:20.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:97480 3.e 3:727a553a:::rbd_data.11214ba95254.0000000000000000:head [delete] snapc 18=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.17520501, 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.94135600100000005, 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:20.591 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97480 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323513+0000", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323528+0000", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5375000000000001e-05 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323552+0000", 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3637000000000001e-05 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323561+0000", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3889999999999994e-06 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323624+0000", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3136e-05 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323633+0000", 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1190000000000003e-06 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261172+0000", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.937538502 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261299+0000", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012696100000000001 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261383+0000", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4559000000000001e-05 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:20.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.262215+0000", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083147699999999995 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264808+0000", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025926870000000002 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264844+0000", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6783999999999999e-05 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264871+0000", 2024-09-21T14:23:20.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6897e-05 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:20.597 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:20.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:20.598 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:20.599+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:20.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:20.698 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:20.699+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:20.742 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:20.743 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:20.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:20.799 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:20.799+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:20.807 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:20.807 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:20.823 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:20.839 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:20.839 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:20.855 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:20.857 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:20.857 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:20.866 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:20.872 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:20.872 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:20.888 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:20.900 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:20.900+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:20.905 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:20.906 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:20.972 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:20.972 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:20.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:21.022 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:21.022+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:21.102 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:21.102+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:21.138 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.138 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:21.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:21.203 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:21.203+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:21.301 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.301 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.304 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:21.304+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:21.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:21.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:21.405+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:21.452 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.453 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.461 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:21.505 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:21.506+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:21.563 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.563 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.571 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:21.613 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:21.613+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:21.707 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:21.707+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:21.714 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.714 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:21.808 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:21.808+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:21.866 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.866 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:21.916 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:21.916+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:21.980 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:21.981 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:21.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:22.010 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:22.010+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:22.091 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.091 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.099 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:22.142 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:22.143+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:22.201 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.201 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:22.252 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:22.252+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:22.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:22.281 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:22.281 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T14:23:22.281 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-21T14:23:22.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-21T14:23:22.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:23:22.312 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:22.313+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:22.367 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.367 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:22.418 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:22.418+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:22.514 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:22.513+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:22.520 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.520 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.528 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:22.615 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:22.614+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:22.630 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.630 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:22.716 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:22.716+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:22.740 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.740 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.749 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:22.832 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:22.833+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:22.890 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:22.890 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:22.898 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:22.940 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:22.940+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:23.018 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:23.018+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:23.040 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:23.041 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:23.049 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:23.132 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:23.133+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:23.152 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:23.192 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:23.192 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:23.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:23.242 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:23.242+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:23.274 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:23.274 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:23:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-21T14:23:23.295 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:23.295 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:23.312 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:23.321 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:23.321+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:23.328 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:23.328 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:23.344 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:23.344 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:23.348 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:23.349 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:23.357 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:23.360 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:23.360 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:23.376 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:23.376 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:23.392 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:23.392 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:23.422 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:23.422+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.462 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865377628, 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:23.463 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.464 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:23.466 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865360114, 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.467 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86534286, 2024-09-21T14:23:23.471 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:23.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:23.474 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86531501, 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.476 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:23.477 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.478 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86529340600001, 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:23.480 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:23.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:23.483 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865276408, 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:23.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:23.488 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865259388, 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.489 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865241938, 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:23.493 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.494 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.495 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86522479999999, 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.500 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.501 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86520157299999, 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.502 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.503 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.504 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.506 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865185299, 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:23.507 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.508 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.509 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86516092700001, 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86513836, 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86512143900001, 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.522 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.523 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.86510493999999, 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.524 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.527 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:23.528 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865083943, 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:23.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:23.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 113.865061885, 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:23.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 90.690991013000001, 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:23.537 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:23.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.540 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:23.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.548757889000001, 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:23.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:23.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:23.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 58.316047812999997, 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:23.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:23.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:23.553 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:23.554 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:23.555+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:23.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:23.655 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:23.656+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:23.657 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:23.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:23.657 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:23.657 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:23.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:23.756 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:23.756+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:23.766 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:23.766 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:23.766 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:23.766 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:23.766 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:23.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:23.857 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:23.857+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.726527306999998, 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.383627844000003, 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.383606505000003, 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.383590697000002, 2024-09-21T14:23:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.383574422999999, 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:23.896 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.383555694999998, 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382586256000003, 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382563623999999, 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:23.912 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382546984999998, 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:23.916 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382531008999997, 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.920 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382513858000003, 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:23.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.924 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382497096000002, 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:23.928 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.382289671999999, 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:23.932 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 50.381944464, 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:23.936 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.536072900000001, 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:23.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:23.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.525533377000002, 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:23.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.520319000000001, 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.950 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.499607415, 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:23.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:23.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:23.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:23.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 46.496534263000001, 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:23.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:97480 3.e 3:727a553a:::rbd_data.11214ba95254.0000000000000000:head [delete] snapc 18=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.5544090659999998, 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.94135600100000005, 2024-09-21T14:23:23.970 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97480 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:23.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323513+0000", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323528+0000", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5375000000000001e-05 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323552+0000", 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3637000000000001e-05 2024-09-21T14:23:23.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323561+0000", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3889999999999994e-06 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323624+0000", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3136e-05 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323633+0000", 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1190000000000003e-06 2024-09-21T14:23:23.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261172+0000", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.937538502 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261299+0000", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012696100000000001 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261383+0000", 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4559000000000001e-05 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.262215+0000", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083147699999999995 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264808+0000", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025926870000000002 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264844+0000", 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6783999999999999e-05 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:23.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264871+0000", 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6897e-05 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:23.976 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:23.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:23.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:23.977+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:24.078 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:24.078+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:24.120 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:24.121 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.129 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:24.179 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:24.180+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:24.271 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:24.271 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:24.272 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:24.272 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:24.280 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:24.280+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:24.380 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:24.380+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:24.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 111.170553871, 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:23:24.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:23:24.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678729408999999, 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:23:24.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678711704999998, 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:23:24.401 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678696033999998, 2024-09-21T14:23:24.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:23:24.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.409 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.67868052, 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:24.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:23:24.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:23:24.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678665452999994, 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.417 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678650258000005, 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:23:24.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:23:24.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:23:24.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678635151999998, 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.422 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678620042000006, 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:24.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:23:24.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:23:24.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678604676999996, 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.430 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:23:24.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:23:24.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678589217999999, 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:23:24.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:23:24.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678574185000002, 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.439 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:23:24.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:23:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678558296000006, 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:23:24.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:23:24.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:24.447 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678532619999999, 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:24.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:23:24.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:23:24.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678517334999995, 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.452 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678495370999997, 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.456 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:24.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 86.678479388, 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.627938, 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.464 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:24.469 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 59.163884244000002, 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.230973259999999, 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:23:24.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:23:24.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.479 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:24.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:24.481+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:24.581 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:24.581+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:24.623 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.623 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:24.623 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:24.623 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:24.681 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:24.682+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:24.775 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.775 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:24.775 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:24.775 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:24.775 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.782 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:24.783+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:24.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:24.883 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:24.883+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.886 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.49377573, 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:24.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:24.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.890 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.493764829, 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.891 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:24.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:24.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.49375442100001, 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.895 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.49374648, 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.417008495, 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.903 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41699008499999, 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.907 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41696433600001, 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:24.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:24.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416943307, 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416917457, 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:24.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41689897800001, 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:24.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.927 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41688085600001, 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:24.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.931 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41686049400001, 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.935 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416840802, 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.939 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41682382400001, 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:24.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416804152, 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:24.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:24.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.947 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41677468899999, 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416732443, 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:24.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:24.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.955 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.41671345500001, 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:24.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.959 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416690097, 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:24.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:24.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:24.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.963 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.416666744, 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:24.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:24.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:24.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:24.967 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:24.968 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:25.012 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:25.012+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:25.069 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:25.069 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:25.069 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:25.069 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:25.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:25.119 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:25.119+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:25.186 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:25.186+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:25.221 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:25.222 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:25.222 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:25.222 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:25.222 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:25.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:25.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:23:25.315 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:25.315+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:25.384 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 135.84227614900001, 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:25.385 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:25.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.391 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996699786999997, 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.395 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996685755000001, 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:25.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.399 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996669756000003, 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:25.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.403 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996654860000007, 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:25.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:25.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.407 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996639923999993, 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:25.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:25.411 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996624944000004, 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:25.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:25.415 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996610328000003, 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:25.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996595280999998, 2024-09-21T14:23:25.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:25.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:25.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996580722000004, 2024-09-21T14:23:25.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:25.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996565997000005, 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:25.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:25.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996551557000004, 2024-09-21T14:23:25.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:25.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996537207000003, 2024-09-21T14:23:25.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:25.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996517140999998, 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:25.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:25.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996502230999994, 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:25.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996487290999994, 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:25.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 97.996467597999995, 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:25.451 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:25.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.348986709000002, 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:25.459 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.238837525999998, 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:25.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:25.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.335274982000001, 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:25.465 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:25.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:25.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:25.471 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:25.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:25.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:25.473 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:25.472+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:25.573 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:25.573+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:25.575 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:25.575 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:25.575 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:25.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:25.631 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:25.631 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:25.647 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:25.647 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:25.663 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:25.663 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:25.673 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:25.673+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:25.678 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:25.678 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:25.694 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:25.695 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:25.710 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:25.711 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:25.727 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:25.727 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:25.728 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:25.728 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:25.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:25.743 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:25.779 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:25.779+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:25.875 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:25.875+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:25.880 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:25.880 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:25.889 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:25.976 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:25.976+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:25.990 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:25.991 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.000 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:26.084 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:26.085+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:26.143 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.143 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.151 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:23:26.152 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:26.194 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:26.194+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:26.279 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:26.279+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:26.303 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.304 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.313 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:26.380 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:26.380+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:26.415 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.415 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.424 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:26.480 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:26.480+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:26.566 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.567 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:26.581 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:26.581+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.677 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.682 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:26.682+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:26.686 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:26.782 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:26.782+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:26.827 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.827 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.836 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:26.883 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:26.883+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:26.936 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:26.937 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:26.945 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:26.984 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:26.984+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:27.046 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.046 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:27.085 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:27.085+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:27.171 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.171 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:27.185 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:27.185+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:27.286 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:27.286+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:27.289 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.290 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:27.386 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:27.386+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:27.441 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.441 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:27.487 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:27.487+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:27.588 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:27.588+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:27.663 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.663 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:27.715 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:27.715+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:27.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:27.774 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:27.774 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:27.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:27.790 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:27.790+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:27.875 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:27.876 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:27.890 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:27.891+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:27.892 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:27.892 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:27.908 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:27.923 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:27.924 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:27.930 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:27.930 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:27.930 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:27.930 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:27.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:27.940 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:27.940 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:27.956 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:27.957 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:27.973 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:27.973 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:27.992 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:27.992+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:28.004 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:28.004 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:28.053 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:28.053 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:28.053 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:28.053 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:28.053 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:28.062 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:28.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:28.093+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:28.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:23:28.210 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:28.210+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:28.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.818415434, 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.81840453300001, 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:28.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.818394125, 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.224 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.818386184, 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:28.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:28.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741648199, 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.229 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:28.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:28.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741629789, 2024-09-21T14:23:28.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:28.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:28.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.74160404, 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:28.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741583011, 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:28.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:28.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.246 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741557161, 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:28.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741538682, 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:28.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.74152056, 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:28.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741500198, 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.264 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741480506, 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:28.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:28.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741463528, 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:28.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:28.273 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741443856, 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741414393, 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.278 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:28.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.74137214700001, 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:28.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741353159, 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:28.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.74132980100001, 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.291 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:28.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:28.295 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 118.741306448, 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:28.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:28.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.300 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.300 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.300 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.300 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.300 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:28.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:28.301 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:28.301+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:28.305 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7] out_osds: [2, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:28.305 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:28.305 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-21T14:23:28.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-21T14:23:28.402 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:28.402+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:28.453 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:28.453 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:28.453 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:28.453 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:28.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:28.503 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:28.503+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:28.566 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:28.567 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:28.567 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:28.567 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:28.567 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:28.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:28.604 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:28.604+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:28.642 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:23:28.642+0000 7f0ce787f640 -1 mon.a@0(leader).osd e141 definitely_dead 0 2024-09-21T14:23:28.678 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:28.678 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 139.136276907, 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:28.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:28.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:28.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43097 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:28.685 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.29070054499999, 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040386960999999999, 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43097 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:28.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387516+0000", 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387515+0000", 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387517+0000", 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.858e-06 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387522+0000", 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1850000000000003e-06 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387526+0000", 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6449999999999998e-06 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427896+0000", 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040369491 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427903+0000", 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1069999999999999e-06 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.689 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290686513, 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:28.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.693 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290670514, 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:28.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290655618, 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:28.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290640682, 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:28.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290625702, 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:28.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:28.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290611086, 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.710 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:28.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.29059603899999, 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:28.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.29058148, 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:28.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290566755, 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:28.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290552315, 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.726 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:28.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290537965, 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:28.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.733 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.29051789899999, 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:28.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290502989, 2024-09-21T14:23:28.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:28.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:28.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.290488049, 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:28.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:28.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 101.29046835600001, 2024-09-21T14:23:28.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.642987466999998, 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:28.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.532838283999993, 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:28.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:28.759 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 63.629275739999997, 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:28.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:28.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:28.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:28.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:28.765 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:28.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:28.767 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:28.766+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:28.867 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:28.867+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:28.870 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:28.870 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:28.870 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:28.870 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:28.879 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:28.968 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:28.969+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:29.022 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.022 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:29.022 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:29.022 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:29.022 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.031 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:29.074 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:29.074+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:29.171 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:29.171+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.409334403000003, 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:29.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.409257893000003, 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.408447929000005, 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:29.190 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.408420871999994, 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407972649000001, 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407950309, 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.199 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:29.203 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407908617999993, 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:29.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:29.207 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407749953000007, 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.208 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:29.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:29.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407717532000007, 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:29.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407689003000002, 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407667907000004, 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:29.225 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407635737000007, 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:29.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.229 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407559247999998, 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407536308000005, 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.234 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:29.238 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407519291, 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.241 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407370438000001, 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407354728000001, 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.247 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.249 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407339304999994, 2024-09-21T14:23:29.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:29.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:29.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:29.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407207240000005, 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.256 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.257 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.407172183, 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:29.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.265 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:29.310 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:29.310+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:29.316 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:23:29.316 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.3. 2024-09-21T14:23:29.337 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-21T14:23:29.372 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:29.372+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:29.377 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:29.377 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:29.377 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:29.473 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:29.474+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:29.528 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.528 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:29.528 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:29.528 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:29.528 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.536 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:29.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:29.580+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:29.640 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.640 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 65.638707775, 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:29.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.142087955000001, 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:29.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.142062385000003, 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.651 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:29.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.141107996999999, 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:29.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.140793496000001, 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.660 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:29.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.139378241999999, 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:29.664 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:29.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.138330791999998, 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:29.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:29.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.137047377999998, 2024-09-21T14:23:29.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:29.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.135726151999997, 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.680 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.134685218000001, 2024-09-21T14:23:29.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:29.688 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.134433778000002, 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:29.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.692 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.133388029000002, 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.696 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.133001086999997, 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.697 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:29.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:29.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.132065503, 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:29.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:29.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.131972834000003, 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:29.705 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:29.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:29.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.709 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.127891615999999, 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:29.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.127708783999999, 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.717 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.127385072000003, 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:29.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.127263618999997, 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.722 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:29.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.725 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 56.126005020000001, 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:29.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:29.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:29.730 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:29.730 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:29.731 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:29.731+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:29.832 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:29.832+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:29.873 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.873 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:29.873 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:29.873 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:29.933 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:29.933+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:29.985 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:29.985 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:29.985 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:29.994 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:30.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:30.038 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:30.038+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665706297, 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.098 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.66568124699999, 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.102 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.103 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.104 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:30.105 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665663095, 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:30.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.107 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.108 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.109 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.110 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.66563985800001, 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:30.112 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:30.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.114 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665618791, 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:30.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:30.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665602431, 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:30.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:30.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.122 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.66557215100001, 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:30.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.126 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665557013, 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.130 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665537417, 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:30.134 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.66552864000001, 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:30.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:30.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665516279, 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.139 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:30.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:30.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665506862, 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.143 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:30.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665485671, 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.151 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665468563, 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.155 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.66545162, 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.156 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:30.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:30.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665435241, 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.160 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.665393881, 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.164 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:30.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 120.6653742, 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:30.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:30.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 97.260485752999998, 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:30.172 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:30.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:30.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:30.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:30.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 37.897714921999999, 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:30.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:30.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:30.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:30.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:30.184 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:30.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:30.185 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:30.185 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:30.185 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:30.186 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:30.186 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:30.186 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:30.187 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:30.185+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:30.192 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:30.192 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:30.209 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:30.209 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:30.225 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:30.225 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:30.241 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:30.241 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:30.256 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:30.257 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:30.285 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:30.285+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:30.335 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:30.335 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:30.344 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:30.386 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:30.386+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:30.486 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:30.486+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:30.487 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:30.487 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:30.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:30.587 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:30.587+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:30.597 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:30.598 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:30.606 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:30.689 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:30.689+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:30.712 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:30.712 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:30.721 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:30.805 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:30.805+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:30.863 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:30.863 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:30.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:30.890 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:30.890+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:30.991 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:30.991+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:31.015 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.015 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:31.091 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:31.092+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:31.127 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.127 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:31.192 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:31.193+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:31.285 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.285 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.293 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:31.293+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:31.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:31.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:23:31.393 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:31.393+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:31.430 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.430 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:31.494 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:31.494+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:31.573 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.573 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:31.624 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:31.624+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:31.695 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:31.695+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:31.732 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.732 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.742 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:31.826 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:31.826+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:31.894 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:31.895 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:31.897 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:31.897+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:31.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:31.997 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:31.998+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:32.050 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:32.051 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:32.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:32.103 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:32.103+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:32.152 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:23:32.177 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:32.177 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:32.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:32.199 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:32.200+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:32.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:32.300 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:32.300+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:32.339 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:32.339 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:32.350 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:32.396 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:32.396 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:32.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:32.401+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:32.415 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:32.415 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:32.431 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:32.431 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:32.448 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:32.448 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:32.458 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:32.458 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:32.466 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:32.466 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:32.467 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:32.482 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:32.483 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:32.499 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:32.499 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:32.509 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:32.509+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:32.515 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:32.515 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:32.602 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:32.602+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:32.612 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:32.612 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:32.612 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:32.613 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:32.613 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:32.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:32.703 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:32.704+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:32.724 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:32.724 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:32.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:32.724 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:32.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.956763549000001, 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.956687039000002, 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.729 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:32.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:32.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:32.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955877075000004, 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:32.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:32.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955850018000007, 2024-09-21T14:23:32.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:32.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955401795, 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.743 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:32.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:32.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955379454999999, 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.748 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.752 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955337764000006, 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:32.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.755 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955179099000006, 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.757 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:32.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.759 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.761 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955146678000006, 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.762 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:32.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:32.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955118149, 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.767 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:32.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:32.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955097053000003, 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:32.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:32.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:32.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.955064883000006, 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:32.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954988393999997, 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:32.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:32.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:32.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:32.785 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954965454000003, 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:32.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:32.788 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:32.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954948436999999, 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:32.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:32.791 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:32.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954799584, 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.794 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:32.795 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:32.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:32.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.798 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954783874, 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:32.799 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:32.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:32.802 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954768451000007, 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:32.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954636386000004, 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.807 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:32.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:32.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.811 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.954601328999999, 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:32.812 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:32.813 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:32.814 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:32.815 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:32.816 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:32.817 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:32.818 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:32.817+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:32.918 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:32.918+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:32.964 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:32.964 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:32.964 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:32.964 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:32.973 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:33.019 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:33.019+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:33.077 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.077 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:33.077 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:33.077 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:33.077 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:33.120 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:33.120+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 69.197780347000005, 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:33.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:33.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.205 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.701160526999999, 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:33.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:33.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.701134957000001, 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:33.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.213 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.700180568999997, 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:33.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:33.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:33.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.699866067999999, 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:33.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:33.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:33.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.221 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.698450813999997, 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.224 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:33.225 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.697403364000003, 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.226 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.227 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.696119950000003, 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.229 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:33.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.694798724000002, 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.233 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.693757789999999, 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.237 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.239 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.240 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.69350635, 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.241 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.242 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.243 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.244 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.692460601000001, 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.245 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.246 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:33.247 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.248 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.692073659000002, 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.691138074999998, 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.253 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.254 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.255 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.691045406000001, 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.257 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.258 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:33.259 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.260 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.686964187999997, 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:33.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.686781355999997, 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:33.265 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.266 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.267 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.686457644000001, 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.269 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:33.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.272 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.686336191000002, 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.273 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 59.685077591999999, 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:33.279 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.280 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.282 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:33.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:33.282+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:33.382 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:33.382+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:33.432 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.432 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:33.433 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:33.433 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:33.483 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:33.483+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:33.584 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:33.584+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:33.585 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.585 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:33.585 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:33.586 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:33.586 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.594 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:33.685 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:33.685+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:33.697 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.697 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:33.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:33.697 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:33.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26513362199999, 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:33.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.265108572, 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26509042000001, 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:33.706 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:33.710 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.265067183, 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:33.714 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26504611599999, 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:33.718 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.265029756, 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264999476, 2024-09-21T14:23:33.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.726 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264984338, 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:33.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.730 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264964742, 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:33.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:33.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:33.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264955965, 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.736 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.737 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.738 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264943604, 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.739 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.741 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.742 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26493418699999, 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.743 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.746 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26491299600001, 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.747 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:33.749 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.750 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264895888, 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.751 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:33.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:33.754 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:33.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26487894500001, 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.760 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.761 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26486256600001, 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.763 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:33.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:33.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.26482120599999, 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.767 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:33.768 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:33.769 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:33.770 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.264801525, 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.771 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 100.85991307800001, 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.777 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.778 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:33.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 41.497142246999999, 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.781 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:33.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:33.786 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:33.787 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:33.789 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:33.788+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:33.889 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:33.889+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:33.892 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:33.892 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:33.892 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:33.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:33.990 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:33.990+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:34.002 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:34.002 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:34.002 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:34.002 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:34.003 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:34.011 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:34.091 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:34.090+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:34.114 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517235881, 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517218367, 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.122 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517201113, 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:34.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:34.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517173263, 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.127 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.51715165900001, 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517134661, 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.135 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:34.139 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517117641, 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:34.143 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517100191, 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:34.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.51708305299999, 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:34.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:34.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.51705982599999, 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.152 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.517043552, 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.156 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:34.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:34.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.51701918000001, 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:34.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:34.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:34.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:34.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:34.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:34.167 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.516996613, 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.168 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:34.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.51697969200001, 2024-09-21T14:23:34.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.516963193, 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.179 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.183 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.516942196, 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:34.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:34.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:34.188 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 124.516920138, 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:34.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 101.342849266, 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:34.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 70.200616142000001, 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:34.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.204 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 68.967906065999998, 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:34.206 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:34.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:34.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:34.211 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:34.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:23:34.213 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:34.213+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:34.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:34.314 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:34.314+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:34.318 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:34.319 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:34.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:34.337 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7, 3] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:34.337 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:34.338 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-21T14:23:34.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-21T14:23:34.415 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:34.416+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:34.434 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:34.434 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:34.435 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:34.435 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:34.435 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:34.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:34.516 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:34.516+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:34.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:34.551 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 66.399054581000001, 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.552 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.557 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.056155118, 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:34.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:34.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:34.562 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.056133779, 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:34.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.566 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.056117970999999, 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.567 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:34.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.056101697000003, 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.571 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:34.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.575 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.056082969000002, 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:34.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.579 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.05511353, 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.580 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.581 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.055090898000003, 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:34.584 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:34.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.055074259000001, 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:34.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:34.592 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.055058283000001, 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:34.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.597 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.055041131999999, 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:34.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:34.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.055024369999998, 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:34.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:34.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:34.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:34.606 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.054816946000003, 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.610 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.054471737999997, 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:34.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:34.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:34.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.208600173999997, 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:34.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:34.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.198060650999999, 2024-09-21T14:23:34.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:34.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:34.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:34.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:34.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.192846273999997, 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:34.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:34.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:34.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:34.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:34.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.636 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.172134689000004, 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:34.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:34.643 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.169061536999997, 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:34.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:97480 3.e 3:727a553a:::rbd_data.11214ba95254.0000000000000000:head [delete] snapc 18=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:34.650 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.226936340000002, 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.94135600100000005, 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97480 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323513+0000", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323528+0000", 2024-09-21T14:23:34.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5375000000000001e-05 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323552+0000", 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3637000000000001e-05 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323561+0000", 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3889999999999994e-06 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323624+0000", 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3136e-05 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323633+0000", 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1190000000000003e-06 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261172+0000", 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.937538502 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261299+0000", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012696100000000001 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261383+0000", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4559000000000001e-05 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.262215+0000", 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083147699999999995 2024-09-21T14:23:34.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264808+0000", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025926870000000002 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264844+0000", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6783999999999999e-05 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264871+0000", 2024-09-21T14:23:34.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6897e-05 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:34.657 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:34.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:34.659 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:34.659 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:34.659 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:34.658+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:34.667 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:34.667 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:34.682 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:34.682 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:34.687 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:23:34.687+0000 7f0ce787f640 -1 mon.a@0(leader).osd e147 definitely_dead 0 2024-09-21T14:23:34.699 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:34.699 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:34.715 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:34.716 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:34.731 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:34.732 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:34.748 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:34.748 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:34.759 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:34.759+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:34.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:34.771 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:34.771 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:34.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:34.864 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:34.864+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:34.921 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:34.921 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:34.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:34.961 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:34.961+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:35.061 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:35.061+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:35.071 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.072 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:35.162 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:35.162+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:35.165 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:23:35.165 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.1. 2024-09-21T14:23:35.182 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-21T14:23:35.230 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.230 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:35.283 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:35.283+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:35.341 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.342 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:35.392 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:35.392+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:35.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:35.465+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:35.494 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.494 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:35.566 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:35.566+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:35.606 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.606 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:35.699 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:35.699+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:35.757 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.757 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:35.767 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:35.768+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:35.868 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:35.869+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:35.909 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:35.909 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:35.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:35.969 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:35.969+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:36.032 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.032 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.041 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:36.083 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:36.083+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:36.170 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:36.170+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:36.189 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.189 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:36.282 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:36.283+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:36.342 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.342 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.351 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:36.371 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:36.372+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:36.472 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:36.472+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:36.494 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.495 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:36.573 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:36.573+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:36.605 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.605 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.614 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:36.674 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:36.674+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:36.716 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.717 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.725 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:36.775 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:36.775+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:36.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:36.868 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.868 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:36.877 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:36.876+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:36.906 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:36.906 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:36.922 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:36.923 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:36.938 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:36.938 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:36.954 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:36.954 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:36.970 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:36.970 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:36.976 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:36.977+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:36.979 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:36.979 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:36.987 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:36.987 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:36.988 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:37.003 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:37.003 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:37.020 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:37.078 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:37.078+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:37.131 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:37.131 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:37.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:37.131 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69920224000001, 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.132 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:37.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69917719, 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:37.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699159038, 2024-09-21T14:23:37.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:37.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:37.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699135801, 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:37.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:37.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69911473400001, 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:37.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:37.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699098374, 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.153 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:37.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699068094, 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:37.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:37.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:37.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699052956, 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69903336, 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:37.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:37.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.169 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.170 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.699024583, 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:37.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:37.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:37.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69901222199999, 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:37.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69900280500001, 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:37.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.698981614, 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.698964506, 2024-09-21T14:23:37.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:37.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.698947563, 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:37.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.194 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.698931184, 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69888982400001, 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.200 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:37.203 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 127.69887014299999, 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.204 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.205 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:37.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:37.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 104.293981696, 2024-09-21T14:23:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.209 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.210 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.211 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:37.212 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 44.931210864999997, 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.214 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.217 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.218 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.219 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.220 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:37.220 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:23:37.221 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:37.222 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:37.221+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:37.321 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:37.321+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:37.366 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:37.367 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:37.367 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:37.367 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:37.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:37.422 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:37.423+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:37.481 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:37.481 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:37.481 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:37.481 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:37.481 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:37.490 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:37.531 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:37.531+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:37.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:37.624+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:37.634 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:37.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037534053, 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.635 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:37.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.638 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03751653899999, 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.639 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:37.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037499285, 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03747143499999, 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.648 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:37.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:37.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037449831, 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:37.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.656 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037432833, 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037415813, 2024-09-21T14:23:37.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:37.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03739836299999, 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:37.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:37.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03738122499999, 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:37.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:37.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037357998, 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.673 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:37.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03734172399999, 2024-09-21T14:23:37.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:37.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.681 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037317352, 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:37.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:37.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037294785, 2024-09-21T14:23:37.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:37.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:37.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037277864, 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.691 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03726136500001, 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.695 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:37.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:37.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:37.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.037240368, 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:37.699 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.703 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 128.03721830999999, 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.706 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.707 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 104.863147438, 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.708 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:37.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:37.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:37.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:37.713 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 73.720914313999998, 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:37.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 72.488204237999994, 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:37.718 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:37.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:37.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:37.722 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:37.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:37.765 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:37.765+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:37.825 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:37.825+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:37.836 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:37.837 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:37.837 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:37.837 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:37.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:37.931 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:37.931+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:37.994 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:37.995 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:38.027 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:38.028+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:38.128 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:38.128+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:38.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 70.006735094000007, 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:38.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:38.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:38.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.663835630999998, 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.165 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.663814291999998, 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:38.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:38.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.663798483999997, 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:38.177 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.663782209999994, 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:38.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.663763481999993, 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:38.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662794043000005, 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:38.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:38.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662771410999994, 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662754772, 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:38.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:38.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:38.198 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662738795999999, 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:38.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662721645000005, 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:38.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662704883000004, 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:38.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662497458999994, 2024-09-21T14:23:38.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:38.215 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.662152250999995, 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:38.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.219 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.816280687000003, 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:38.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:38.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:38.226 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.805741163999997, 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.800526787000003, 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:38.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:38.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.779815202000002, 2024-09-21T14:23:38.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:38.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:38.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:38.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 60.776742050000003, 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:97480 3.e 3:727a553a:::rbd_data.11214ba95254.0000000000000000:head [delete] snapc 18=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 23.834616853, 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.94135600100000005, 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97480 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323513+0000", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323528+0000", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5375000000000001e-05 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323552+0000", 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3637000000000001e-05 2024-09-21T14:23:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323561+0000", 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3889999999999994e-06 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323624+0000", 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3136e-05 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323633+0000", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1190000000000003e-06 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261172+0000", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.937538502 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261299+0000", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012696100000000001 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261383+0000", 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4559000000000001e-05 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.262215+0000", 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083147699999999995 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264808+0000", 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025926870000000002 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264844+0000", 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6783999999999999e-05 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264871+0000", 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6897e-05 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.261 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:38.263 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:38.263+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:23:38.363 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:38.363+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:38.418 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:99856 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-21T14:23:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:38.373330+0000", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.045887781000000002, 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045902336000000002, 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "waiting for peered", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99856 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373330+0000", 2024-09-21T14:23:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373330+0000", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373328+0000", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999986 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373340+0000", 2024-09-21T14:23:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1137e-05 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373351+0000", 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1371e-05 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373356+0000", 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0259999999999998e-06 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373396+0000", 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0469999999999997e-05 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:38.373403+0000", 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.156e-06 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-09-21T14:23:38.422 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:38.471 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:38.471+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:38.532 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:38.533 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:38.566 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:38.566+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:38.666 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:38.667+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 125.46778066100001, 2024-09-21T14:23:38.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975956199, 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:23:38.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:23:38.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97593849499999, 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:38.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:23:38.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:23:38.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:38.702 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97592282399999, 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:23:38.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:23:38.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.706 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97590731, 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:38.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:23:38.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:23:38.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975892243, 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:38.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:23:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975877048, 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:23:38.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97586194199999, 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:38.723 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975846832, 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:23:38.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.727 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97583146700001, 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:23:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:23:38.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975816008, 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.732 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:23:38.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975800975, 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:23:38.736 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.740 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975785086, 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:23:38.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:23:38.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97575940999999, 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.745 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:23:38.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97574412500001, 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:23:38.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:23:38.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.97572216099999, 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:23:38.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:23:38.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 100.975706178, 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.758 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:23:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:38.762 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.925164789999997, 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:38.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:23:38.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:38.768 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.461111033999998, 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:23:38.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:23:38.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 70.528200049999995, 2024-09-21T14:23:38.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:23:38.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:38.779 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:38.779 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:38.781 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:38.781+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:38.880 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:38.881+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:38.891 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:38.891 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:38.891 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:38.891 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:38.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:38.981 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:38.981+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:39.007 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:39.007 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:39.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:39.032 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:39.082 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:39.083+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.770307438, 2024-09-21T14:23:39.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.77029653700001, 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:39.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.770286129, 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.172 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:39.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.176 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.77027818799999, 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:39.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:39.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:39.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693540203, 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:39.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:39.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693521793, 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:39.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:39.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:39.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693496044, 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:39.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:39.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69347501499999, 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:39.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:39.198 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693449165, 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:39.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:39.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693430686, 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693412564, 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:39.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69339220200001, 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:39.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:39.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69337250999999, 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:39.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693355532, 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:39.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69333585999999, 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:39.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:39.229 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69330639699999, 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:39.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69326415099999, 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.234 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:39.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.693245163, 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:39.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:39.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69322180500001, 2024-09-21T14:23:39.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:39.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 129.69319845199999, 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:39.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:39.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:39.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:39.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:39.251 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:39.252 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:39.253 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:39.253 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:39.253 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:39.254 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:39.254 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:39.254 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:39.254 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:39.255 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:39.255 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:39.252+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:39.257 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:39.257 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:39.274 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:39.275 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:39.291 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:39.353 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:39.354+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:39.363 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:39.363 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:39.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:39.454 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:39.454+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:39.478 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:39.478 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:39.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:39.555 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:39.556+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:39.588 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:39.588 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:39.597 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:39.657 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:39.657+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:39.711 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:39.711 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:39.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:39.763 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:39.763+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:39.859 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:39.859+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:39.864 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:39.864 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:39.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:39.959 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:39.959+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:40.015 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.015 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.025 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:40.060 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:40.060+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:40.127 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.127 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:40.179 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:40.179+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:40.182 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 6, 7, 3, 1] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:23:40.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:23:40.237 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.237 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:40.262 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:40.262+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:40.362 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:40.362+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:40.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:23:40.391 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.391 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:40.463 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:40.463+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:40.548 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.548 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:40.601 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:40.601+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:40.659 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.659 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.664 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:40.664+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:40.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:40.766 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:40.766+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:40.810 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.811 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.819 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:40.866 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:40.866+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:40.921 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:40.921 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:40.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:40.968 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:40.968+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:41.068 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:41.069+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:41.073 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:41.073 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:41.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:41.154 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:23:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-21T14:23:41.169 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:41.169+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:41.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:23:41.241 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:41.241 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:41.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:41.299 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:41.299+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:41.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:41.371 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:41.371+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:41.373 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:41.374 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:41.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:41.430 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:41.430 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:41.448 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:41.448 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:41.466 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:41.466 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:41.471 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:41.471+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:41.482 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:41.498 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:41.515 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:41.515 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:41.530 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:41.530 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:41.530 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:41.530 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:41.530 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:41.532 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:41.532 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:41.539 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:41.549 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:41.549 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:41.571 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:41.572+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:41.651 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83058 3.14 3:2e2e66e7:::rbd_header.112185aab2af:head [watch reconnect cookie 140451941139520 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.499445461999997, 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.018496587, 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.652 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83058 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151397+0000", 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151394+0000", 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151406+0000", 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1319e-05 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151450+0000", 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4240999999999999e-05 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151462+0000", 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151547+0000", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4252000000000006e-05 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.151570+0000", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3804000000000001e-05 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158104+0000", 2024-09-21T14:23:41.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065336470000000001 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.158115+0000", 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165102+0000", 2024-09-21T14:23:41.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0069872879999999 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165303+0000", 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000200936 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.169894+0000", 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045902429999999999 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84606 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.156545999000002, 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053397212999999999, 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84606 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.659 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494297+0000", 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494296+0000", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494302+0000", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6189999999999999e-06 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494308+0000", 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4479999999999999e-06 2024-09-21T14:23:41.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494314+0000", 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2989999999999996e-06 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547388+0000", 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053074866999999998 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547497+0000", 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108546 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547694+0000", 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019676200000000001 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84607 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29884416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:41.663 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.156524660000002, 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053694156999999999, 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84607 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494318+0000", 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494317+0000", 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494320+0000", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494326+0000", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494330+0000", 2024-09-21T14:23:41.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1559999999999997e-06 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547724+0000", 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053394158999999997 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.547824+0000", 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000100373 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548012+0000", 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000187586 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84609 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30932992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.156508852000002, 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053920023999999997, 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84609 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494334+0000", 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494333+0000", 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494336+0000", 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2450000000000001e-06 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494341+0000", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7320000000000003e-06 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494346+0000", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548047+0000", 2024-09-21T14:23:41.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053701058000000003 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548120+0000", 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3402e-05 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548254+0000", 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013358399999999999 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84610 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31981568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.156492577999998, 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054207439000000003, 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84610 2024-09-21T14:23:41.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494350+0000", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494352+0000", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2520000000000002e-06 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494359+0000", 2024-09-21T14:23:41.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6830000000000002e-06 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494365+0000", 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0039999999999996e-06 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548276+0000", 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053911336999999997 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548375+0000", 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9549999999999994e-05 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548557+0000", 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181953 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84611 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:41.677 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.156473849999998, 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054369747000000003, 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84611 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:41.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494369+0000", 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494368+0000", 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494370+0000", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.058e-06 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494377+0000", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.494381+0000", 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:41.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548572+0000", 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054190956999999998 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548627+0000", 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5059000000000003e-05 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548738+0000", 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011124399999999999 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84710 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 25952256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155504410999995, 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053719754000000002, 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.682 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84710 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495338+0000", 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495341+0000", 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.766e-06 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495349+0000", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495355+0000", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548770+0000", 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053414389999999999 2024-09-21T14:23:41.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.548873+0000", 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010363500000000001 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549058+0000", 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018445899999999999 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84712 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 27000832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155481778999999, 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053901993000000002, 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84712 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.687 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495361+0000", 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495360+0000", 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495363+0000", 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3300000000000001e-06 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495369+0000", 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9440000000000002e-06 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495373+0000", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549090+0000", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053716795999999997 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549148+0000", 2024-09-21T14:23:41.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7747000000000001e-05 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549263+0000", 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504499999999999 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84714 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 28049408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:41.691 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155465140000004, 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054196635999999999, 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84714 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:41.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495377+0000", 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495379+0000", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.165e-06 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495385+0000", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.891e-06 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495389+0000", 2024-09-21T14:23:41.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3270000000000002e-06 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549292+0000", 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053902393 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549395+0000", 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010278899999999999 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549574+0000", 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017942000000000001 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84716 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 29097984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155449164000004, 2024-09-21T14:23:41.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054409198999999998, 2024-09-21T14:23:41.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84716 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:41.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495393+0000", 2024-09-21T14:23:41.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495395+0000", 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495402+0000", 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3709999999999996e-06 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495406+0000", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5920000000000002e-06 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549608+0000", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054201602000000002 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549674+0000", 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5866999999999998e-05 2024-09-21T14:23:41.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549803+0000", 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012886899999999999 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84718 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 30146560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.703 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155432012999995, 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054726207999999998, 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84718 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495410+0000", 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495412+0000", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.322e-06 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495419+0000", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495423+0000", 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6439999999999999e-06 2024-09-21T14:23:41.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549837+0000", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054413805000000003 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.549944+0000", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106579 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550137+0000", 2024-09-21T14:23:41.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019315399999999999 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84720 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155415250999994, 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054942814, 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84720 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495427+0000", 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495429+0000", 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0080000000000001e-06 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495435+0000", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495439+0000", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.442e-06 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550168+0000", 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054728756000000003 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550230+0000", 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2640999999999999e-05 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550370+0000", 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013967600000000001 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84722 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.155207826999998, 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.055045734999999998, 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84722 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495635+0000", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495634+0000", 2024-09-21T14:23:41.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495638+0000", 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.67e-06 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495647+0000", 2024-09-21T14:23:41.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495653+0000", 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4e-06 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550401+0000", 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054747822000000002 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550502+0000", 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010160199999999999 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550680+0000", 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017815499999999999 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.717 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:84724 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.154862618999999, 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054881931000000002, 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.718 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 84724 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495980+0000", 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495979+0000", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495989+0000", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6849999999999996e-06 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.495997+0000", 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3659999999999992e-06 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.496003+0000", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000003e-06 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550697+0000", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.05469367 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550755+0000", 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8433000000000001e-05 2024-09-21T14:23:41.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:33.550862+0000", 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106346 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86301 3.13 3:c8aca627:::rbd_data.1121d8e786d8.000000000000006b:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.308991055000007, 2024-09-21T14:23:41.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9707460540000001, 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86301 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341851+0000", 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341850+0000", 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341857+0000", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2230000000000004e-06 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341871+0000", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4e-05 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341876+0000", 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.366e-06 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341925+0000", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9082999999999997e-05 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.341936+0000", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0579e-05 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222961+0000", 2024-09-21T14:23:41.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.88102527799999997 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222968+0000", 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9059999999999996e-06 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307235+0000", 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.084267219 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307328+0000", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.289e-05 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307447+0000", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119358 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308890+0000", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014432469999999999 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312487+0000", 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035961270000000002 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312521+0000", 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4289999999999999e-05 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312598+0000", 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6650999999999995e-05 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.729 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86313 3.13 3:c93627bc:::rbd_data.1121d8e786d8.00000000000000d4:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.298451532000001, 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.960387506, 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86313 2024-09-21T14:23:41.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352391+0000", 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352388+0000", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352402+0000", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.363e-05 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352427+0000", 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5233999999999999e-05 2024-09-21T14:23:41.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352439+0000", 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1815e-05 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352503+0000", 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4197999999999998e-05 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.352516+0000", 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2951e-05 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.222997+0000", 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.87048026599999995 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223004+0000", 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9970000000000001e-06 2024-09-21T14:23:41.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.307913+0000", 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.08490975 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308019+0000", 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106017 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308146+0000", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012679100000000001 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312635+0000", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044888810000000001 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312676+0000", 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1449999999999998e-05 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312705+0000", 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8374999999999999e-05 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312778+0000", 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3700000000000002e-05 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86319 3.13 3:c82ef95d:::rbd_data.1121d8e786d8.0000000000000111:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.293237155, 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.955280055, 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86319 2024-09-21T14:23:41.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357605+0000", 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357604+0000", 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357610+0000", 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0759999999999999e-06 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357621+0000", 2024-09-21T14:23:41.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0806e-05 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357627+0000", 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357673+0000", 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5884000000000003e-05 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.357683+0000", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0251999999999999e-05 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223031+0000", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.86534784899999995 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223039+0000", 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.644e-06 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.308915+0000", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0858760080000001 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309011+0000", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6506000000000001e-05 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309124+0000", 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112726 2024-09-21T14:23:41.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312668+0000", 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035433000000000001 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312787+0000", 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011937199999999999 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312814+0000", 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6987000000000001e-05 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312885+0000", 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1538000000000005e-05 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86340 3.13 3:c966d4c5:::rbd_data.1121d8e786d8.000000000000026c:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.272525569999999, 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9363309850000001, 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86340 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378317+0000", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378316+0000", 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378321+0000", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7879999999999997e-06 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378330+0000", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4839999999999993e-06 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378335+0000", 2024-09-21T14:23:41.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6099999999999999e-06 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378374+0000", 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9434000000000002e-05 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.378385+0000", 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223059+0000", 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84467365299999997 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223066+0000", 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3420000000000002e-06 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309670+0000", 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0866041129999999 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309759+0000", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9242999999999999e-05 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.309872+0000", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112497 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312922+0000", 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030499170000000001 2024-09-21T14:23:41.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314481+0000", 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015595940000000001 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314533+0000", 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1718000000000002e-05 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.314648+0000", 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011471 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:86343 3.13 3:cfdac2b1:::rbd_data.1121d8e786d8.000000000000026f:head [delete] snapc 31=[31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.269452418, 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.935720983, 2024-09-21T14:23:41.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 86343 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:41.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381390+0000", 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381389+0000", 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381394+0000", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.51e-06 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381404+0000", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0434999999999999e-05 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381409+0000", 2024-09-21T14:23:41.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1540000000000001e-06 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381455+0000", 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5556000000000002e-05 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:37.381465+0000", 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0171e-05 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223086+0000", 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.84162087600000002 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:38.223094+0000", 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6469999999999998e-06 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.312948+0000", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0898539460000001 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313044+0000", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6114999999999997e-05 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313165+0000", 2024-09-21T14:23:41.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000121339 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.313978+0000", 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081341300000000003 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.316961+0000", 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029823839999999998 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317009+0000", 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7902999999999999e-05 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:39.317111+0000", 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010233100000000001 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:97480 3.e 3:727a553a:::rbd_data.11214ba95254.0000000000000000:head [delete] snapc 18=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T14:23:41.761 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.327327221000001, 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.94135600100000005, 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97480 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:41.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323515+0000", 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323513+0000", 2024-09-21T14:23:41.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323528+0000", 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5375000000000001e-05 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323552+0000", 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3637000000000001e-05 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323561+0000", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3889999999999994e-06 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323624+0000", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3136e-05 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:14.323633+0000", 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1190000000000003e-06 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261172+0000", 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.937538502 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261299+0000", 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012696100000000001 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:41.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.261383+0000", 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4559000000000001e-05 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.262215+0000", 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083147699999999995 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264808+0000", 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025926870000000002 2024-09-21T14:23:41.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264844+0000", 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6783999999999999e-05 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:15.264871+0000", 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6897e-05 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:41.768 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:41.769 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:41.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:41.771 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:41.770+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:41.871 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:41.871+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:41.872 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:41.872 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:41.873 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:41.873 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:41.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:41.971 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:41.971+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:42.022 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.022 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:42.022 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:42.022 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:42.023 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:42.072 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:42.072+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:42.157 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:23:42.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:23:42.173 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:42.173+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:21847 3.d 3:b18ea582:::rbd_object_map.112183cd9830:head [read 0~86] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-21T14:23:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 128.96249396299999, 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.008183426, 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21847 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:42.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220937+0000", 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220936+0000", 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.999999 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220941+0000", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1760000000000001e-06 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220951+0000", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8020000000000003e-06 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220956+0000", 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.220998+0000", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2796999999999997e-05 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:33.221005+0000", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3010000000000003e-06 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228715+0000", 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0077100189999999 2024-09-21T14:23:42.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.228829+0000", 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113922 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:34.229120+0000", 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000291704 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53669 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470669501, 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236642000000003, 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53669 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712761+0000", 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712763+0000", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.666e-06 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712770+0000", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5830000000000003e-06 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712775+0000", 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9520000000000002e-06 2024-09-21T14:23:42.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744842+0000", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032066900000000002 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744926+0000", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4101e-05 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.744998+0000", 2024-09-21T14:23:42.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1982000000000003e-05 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53671 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470651797, 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032396466999999998, 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53671 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712779+0000", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712781+0000", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2050000000000001e-06 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712786+0000", 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5310000000000001e-06 2024-09-21T14:23:42.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712791+0000", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4909999999999999e-06 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745028+0000", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032236961000000001 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745115+0000", 2024-09-21T14:23:42.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6791000000000003e-05 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745176+0000", 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0933000000000001e-05 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53672 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470636126, 2024-09-21T14:23:42.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032550623000000001, 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53672 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712795+0000", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712794+0000", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712796+0000", 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.945e-06 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712802+0000", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.536e-06 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712806+0000", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745202+0000", 2024-09-21T14:23:42.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032395321999999997 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745280+0000", 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8427e-05 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745345+0000", 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5294000000000002e-05 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53673 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470620612, 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032717824, 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53673 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712810+0000", 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712812+0000", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8649999999999999e-06 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712817+0000", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4040000000000003e-06 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712821+0000", 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3039999999999998e-06 2024-09-21T14:23:42.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745374+0000", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032552642999999999 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745461+0000", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6520999999999994e-05 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745528+0000", 2024-09-21T14:23:42.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7392999999999997e-05 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53963 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470605545, 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032888964, 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.207 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53963 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712825+0000", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712827+0000", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712832+0000", 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4920000000000001e-06 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712837+0000", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999999e-06 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745555+0000", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032718574 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745643+0000", 2024-09-21T14:23:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7509000000000005e-05 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745714+0000", 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1348999999999997e-05 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53964 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.47059034999999, 2024-09-21T14:23:42.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033060355999999999, 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53964 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712840+0000", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712842+0000", 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.781e-06 2024-09-21T14:23:42.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712847+0000", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712852+0000", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2710000000000003e-06 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745742+0000", 2024-09-21T14:23:42.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032890317000000002 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745833+0000", 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1380999999999998e-05 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745901+0000", 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7385000000000006e-05 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53965 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470575244, 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238106000000003, 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53965 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712856+0000", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712855+0000", 2024-09-21T14:23:42.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712857+0000", 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712863+0000", 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4550000000000003e-06 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712867+0000", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3630000000000004e-06 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.745930+0000", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033062564000000003 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746024+0000", 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4016000000000002e-05 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746094+0000", 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0166999999999994e-05 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53966 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470560134, 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033409686000000001, 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53966 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712871+0000", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712870+0000", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712872+0000", 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.018e-06 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712878+0000", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712882+0000", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4730000000000002e-06 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746121+0000", 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033238704000000001 2024-09-21T14:23:42.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746211+0000", 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9779999999999998e-05 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746280+0000", 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9661999999999998e-05 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.224 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53969 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470544769, 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033581837000000003, 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53969 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:42.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712886+0000", 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712888+0000", 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0590000000000001e-06 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712893+0000", 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5770000000000001e-06 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712898+0000", 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4100000000000001e-06 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746310+0000", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033411911000000002 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746399+0000", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9308999999999995e-05 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746468+0000", 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8881999999999995e-05 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53970 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.47052931, 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033755878000000003, 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53970 2024-09-21T14:23:42.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712902+0000", 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712901+0000", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712903+0000", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.855e-06 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712909+0000", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712913+0000", 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2610000000000004e-06 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746498+0000", 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033585509 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746585+0000", 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7093000000000001e-05 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746657+0000", 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1952000000000005e-05 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53971 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.47051427700001, 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033927006000000003, 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53971 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712917+0000", 2024-09-21T14:23:42.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712916+0000", 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712918+0000", 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1449999999999998e-06 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712924+0000", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5049999999999998e-06 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712928+0000", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4970000000000003e-06 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746686+0000", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033757721999999997 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746775+0000", 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8540000000000003e-05 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746844+0000", 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8911999999999993e-05 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53984 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470498388, 2024-09-21T14:23:42.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034098077999999997, 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53984 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712932+0000", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712943+0000", 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1178999999999999e-05 2024-09-21T14:23:42.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712949+0000", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0920000000000003e-06 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712954+0000", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6639999999999997e-06 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746873+0000", 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033919289999999998 2024-09-21T14:23:42.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.746960+0000", 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6564999999999995e-05 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747031+0000", 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0837e-05 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53985 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470472712, 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034257700000000002, 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53985 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712958+0000", 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712960+0000", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9889999999999999e-06 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712965+0000", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712970+0000", 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3209999999999998e-06 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747060+0000", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034090588999999998 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747147+0000", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7046999999999995e-05 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747216+0000", 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8678999999999998e-05 2024-09-21T14:23:42.245 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53986 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470457427, 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034425077999999998, 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53986 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712973+0000", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712975+0000", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1169999999999998e-06 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712981+0000", 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712991+0000", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8220000000000002e-06 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747245+0000", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034253582999999997 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747331+0000", 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6681000000000006e-05 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747398+0000", 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7163999999999996e-05 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53987 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.470435463, 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034587332999999998, 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.250 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53987 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712995+0000", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.712997+0000", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.295e-06 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713003+0000", 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.784e-06 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713007+0000", 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4120000000000003e-06 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747426+0000", 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034418532000000002 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747515+0000", 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8648999999999998e-05 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747583+0000", 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8031e-05 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:53989 3.10 3:0b8fd390:::rbd_data.1121c4f3f6c2.0000000000000017:19 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 104.47041948, 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034765499999999998, 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53989 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713011+0000", 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713013+0000", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713018+0000", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.713023+0000", 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3189999999999996e-06 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747612+0000", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034589386999999999 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747704+0000", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2478999999999993e-05 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:57.747777+0000", 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2373999999999995e-05 2024-09-21T14:23:42.258 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:80476 3.1a 3:5af48bc1:::rbd_data.1121590fed9b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 78.419878092000005, 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086203170999999995, 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 80476 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763553+0000", 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.763550+0000", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999971 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782807+0000", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019256905000000001 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782841+0000", 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4153999999999999e-05 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782872+0000", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1272999999999997e-05 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.782921+0000", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8492999999999998e-05 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.783055+0000", 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013407299999999999 2024-09-21T14:23:42.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.798713+0000", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015658076999999999 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.843739+0000", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045026346000000002 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849653+0000", 2024-09-21T14:23:42.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059139020000000004 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849707+0000", 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3316000000000002e-05 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:23.849756+0000", 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9255000000000002e-05 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82816 3.1c 3:3dbf87d1:::rbd_data.1121a2b38c70.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 76.955824336000006, 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.064051774000000006, 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82816 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227607+0000", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.227604+0000", 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:42.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236897+0000", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0092929329999999997 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236923+0000", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5211999999999998e-05 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.236932+0000", 2024-09-21T14:23:42.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.628e-06 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237016+0000", 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4246000000000006e-05 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.237141+0000", 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012431899999999999 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 0", 2024-09-21T14:23:42.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.252115+0000", 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01497387 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291566+0000", 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039451081999999998 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291574+0000", 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291612+0000", 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7858999999999998e-05 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.291658+0000", 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6076999999999999e-05 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:83060 3.d 3:b76f9a1b:::rbd_data.1121590fed9b.0000000000000005:head [sparse-read 0~18874368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:23:42.270 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.022913352000003, 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.00504207, 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 83060 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160517+0000", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160515+0000", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160528+0000", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2994e-05 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160549+0000", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1162000000000002e-05 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160561+0000", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2062000000000001e-05 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160608+0000", 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6734999999999999e-05 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:28.160619+0000", 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0920999999999999e-05 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165467+0000", 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0048478890000001 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:29.165560+0000", 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2740999999999999e-05 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.275 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:42.277 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:42.276+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:42.377 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:42.377+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:42.421 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.421 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:42.421 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:42.421 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:42.478 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:42.479+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:42.571 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.572 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:42.572 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:42.572 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:42.572 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.579 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:42.579+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:42.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:42.681 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:42.681+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.33138247799999, 2024-09-21T14:23:42.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:42.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:42.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.331371577, 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:42.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.733 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.33136116899999, 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:42.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.33135322800001, 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:42.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25461524299999, 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:42.742 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25459683299999, 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.747 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:42.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:42.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25457108399999, 2024-09-21T14:23:42.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:42.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:42.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25455005500001, 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:42.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:42.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.254524205, 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.760 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:42.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:42.764 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25450572599999, 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.768 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25448760399999, 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:42.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.254467242, 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.773 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:42.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25444755000001, 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:42.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:42.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:42.782 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25443057199999, 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:42.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:42.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25441090000001, 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.787 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25438143700001, 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:42.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25433919100001, 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.796 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:42.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25432020299999, 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:42.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:42.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:42.805 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.254296845, 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:42.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 133.25427349200001, 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:42.810 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:42.814 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:42.815 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:42.816 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:42.816+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:42.917 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:42.917+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:42.919 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:42.919 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:42.919 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:42.919 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:42.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:43.018 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:43.018+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:43.070 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:43.070 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:43.070 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:43.071 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:43.071 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:43.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:43.118 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:43.119+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:43.156 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:23:43.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:23:43.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:23:43.191 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:43.191 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:43.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 153.64901008199999, 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:43.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:43.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:43.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:43.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80341968800001, 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:43.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80340368900001, 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:43.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:43.207 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803388793, 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:43.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:43.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803373857, 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:43.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:43.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80335887699999, 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:43.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80334426100001, 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803329214, 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:43.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:43.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80331465499999, 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80329992999999, 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:43.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:43.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80328548999999, 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.238 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:43.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.80327114000001, 2024-09-21T14:23:43.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:43.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803251074, 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:43.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803236164, 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:43.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803221224, 2024-09-21T14:23:43.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:43.259 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 115.803201531, 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:43.263 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 79.155720642000006, 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 79.045571459000001, 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:43.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:43.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:43.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 78.142008915000005, 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.275 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:43.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:43.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:43.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:99795 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9338455989999996, 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91399163500000002, 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99795 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.281 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:43.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257103+0000", 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257108+0000", 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.374e-06 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257116+0000", 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2900000000000002e-06 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257120+0000", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5709999999999999e-06 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257160+0000", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9406000000000002e-05 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257167+0000", 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.781e-06 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162276+0000", 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.905108939 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162390+0000", 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114109 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7", 2024-09-21T14:23:43.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162524+0000", 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013418600000000001 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.163436+0000", 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091262600000000004 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.170942+0000", 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0075057079999999998 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.171067+0000", 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124475 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.171095+0000", 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-05 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:43.287 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:43.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:43.290 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:43.289+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:43.390 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:43.390+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:43.404 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop_reply(client.4385.0:100469 3.8 e156/19 ondisk, result = 0)", 2024-09-21T14:23:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:23:43.405234+0000", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 8.6731999999999996e-05, 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000100775, 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405234+0000", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405234+0000", 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405234+0000", 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405238+0000", 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2189999999999998e-06 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405247+0000", 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6340000000000007e-06 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405251+0000", 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405284+0000", 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2709999999999997e-05 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405292+0000", 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1189999999999997e-06 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:100469 3.8 3:137995e8:::rbd_header.1121cf46bdbd:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:23:43.404133+0000", 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 0.0011879150000000001, 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001242008, 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 100469 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404133+0000", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404133+0000", 2024-09-21T14:23:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404132+0000", 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404137+0000", 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4229999999999998e-06 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404148+0000", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0798999999999999e-05 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404153+0000", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.95e-06 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404193+0000", 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0595e-05 2024-09-21T14:23:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404273+0000", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9458999999999999e-05 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 7", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.404438+0000", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016559699999999999 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405001+0000", 2024-09-21T14:23:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00056316000000000005 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405299+0000", 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00029716299999999997 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405343+0000", 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4511999999999997e-05 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:23:43.405375+0000", 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.2128999999999997e-05 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 2 2024-09-21T14:23:43.414 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:43.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:43.491 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:43.491+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:43.524 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:43.525 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:43.525 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:43.525 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:43.525 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:43.534 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:43.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:23:43.604 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:43.604+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:43.637 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.869693839000007, 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.641 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.642 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.869617328999993, 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.643 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.644 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.645 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868807364999995, 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.647 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.648 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:43.650 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.651 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868780307999998, 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.652 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.653 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:43.655 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.656 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868332085000006, 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.657 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:43.658 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.659 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.660 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:43.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868309745000005, 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.662 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:43.664 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.665 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868268053999998, 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:43.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.668 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.669 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.670 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.671 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868109388999997, 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:43.672 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.673 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:43.674 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.675 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868076967999997, 2024-09-21T14:23:43.676 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:43.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:43.679 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.680 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868048439000006, 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.681 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.683 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.685 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.868027342999994, 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.686 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.687 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:43.689 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.690 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867995172999997, 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.691 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.692 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:43.694 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.695 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867918684000003, 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.696 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.698 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:43.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.700 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867895743999995, 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.701 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:43.702 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.705 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867878727000004, 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.706 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.707 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.708 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.709 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:43.710 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867729874000005, 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.711 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:43.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.714 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867714164000006, 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:43.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.717 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:43.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867698740999998, 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.721 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:43.722 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.723 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.725 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867566675999996, 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.726 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.729 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:43.730 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.867531619000005, 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.731 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:43.732 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.733 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:43.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:43.735 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:43.736 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:43.736 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:43.737 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:43.737 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:43.738 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:43.738 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:43.737+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:43.738 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:43.738 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:43.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:43.739 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:43.742 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:43.742 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:43.758 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:43.758 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:43.775 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:43.775 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:43.791 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:43.791 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:43.843 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:43.843+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:43.881 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:43.881 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:43.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:43.938 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:43.939+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:43.992 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:43.992 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.001 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:44.039 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:44.039+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:44.140 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:44.140+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:44.143 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.143 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.151 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:44.241 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:44.241+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:44.293 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.293 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.302 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:44.342 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:44.342+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-21T14:23:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:23:44.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:23:44.408 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.408 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.418 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:44.444 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:44.444+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:44.529 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.529 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:44.544 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:44.544+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:44.640 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.640 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.645 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:44.646+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:44.649 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:44.746 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:44.746+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:44.791 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.791 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:44.848 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:44.848+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:44.942 INFO:teuthology.orchestra.run.smithi161.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:44.942 INFO:teuthology.orchestra.run.smithi161.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:44.948 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:44.948+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:44.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:23:45.049 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-21T14:23:45.049+0000 7ff22c032640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33504) UID: 0 2024-09-21T14:23:45.055 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.055 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:23:45.150 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-09-21T14:23:45.150+0000 7f8aaa047640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33039) UID: 0 2024-09-21T14:23:45.205 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.205 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:23:45.257 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:45.257+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:45.317 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.317 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:23:45.351 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:45.351+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:45.402 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:23:45.417 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T14:23:45.459 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-21T14:23:45.459+0000 7f7fc9250640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33517) UID: 0 2024-09-21T14:23:45.482 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.483 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:23:45.576 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-21T14:23:45.576+0000 7f462f094640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33510) UID: 0 2024-09-21T14:23:45.638 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.638 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:23:45.653 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:45.654+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:45.754 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:45.754+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:45.790 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.790 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.799 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:23:45.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:23:45.883 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-09-21T14:23:45.883+0000 7ff33a2d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33522) UID: 0 2024-09-21T14:23:45.945 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-21T14:23:45.945 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:23:45.954 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:23:45.956 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:45.957+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:45.960 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-21T14:23:45.961 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:23:45.977 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-21T14:23:45.977 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:23:45.993 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-21T14:23:45.994 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:23:46.009 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-21T14:23:46.009 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:23:46.026 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-21T14:23:46.026 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:23:46.042 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-21T14:23:46.043 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:23:46.058 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-09-21T14:23:46.058+0000 7f09e474d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33046) UID: 0 2024-09-21T14:23:46.059 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-21T14:23:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:23:46.064 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:46.065 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:46.065 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:46.065 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:46.065 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:46.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:23:46.076 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-21T14:23:46.077 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:23:46.155 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T14:23:46.159 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-09-21T14:23:46.159+0000 7f57c342c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33042) UID: 0 2024-09-21T14:23:46.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7453 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:46.189 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.79637112200001, 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019979171, 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7453 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392222+0000", 2024-09-21T14:23:46.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392223+0000", 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.189e-06 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392227+0000", 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.811e-06 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392230+0000", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412087+0000", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857723000000001 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412133+0000", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412201+0000", 2024-09-21T14:23:46.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8181000000000003e-05 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7454 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.79636022099999, 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020111355000000001, 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.194 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7454 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392233+0000", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392234+0000", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392238+0000", 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0099999999999997e-06 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392241+0000", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2559999999999998e-06 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412235+0000", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019993339999999998 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412290+0000", 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4966000000000002e-05 2024-09-21T14:23:46.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412344+0000", 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4848000000000001e-05 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7455 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.79634981300001, 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020274342000000001, 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7455 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392243+0000", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392244+0000", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.029e-06 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392247+0000", 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-06 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392249+0000", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3080000000000002e-06 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412381+0000", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020131468 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412458+0000", 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7241000000000001e-05 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412518+0000", 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9633999999999999e-05 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:7456 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.796341872, 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02046895, 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.202 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 7456 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392251+0000", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967296 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392252+0000", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.088e-06 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392255+0000", 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7190000000000001e-06 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.392257+0000", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4399999999999999e-06 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412544+0000", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020287001999999998 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412623+0000", 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8380000000000005e-05 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.412720+0000", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7549999999999999e-05 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8253 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 460800~675] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71960388700001, 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019861740999999999, 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8253 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468989+0000", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468991+0000", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3530000000000002e-06 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.468998+0000", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2430000000000002e-06 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469002+0000", 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488667+0000", 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019664942000000001 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488749+0000", 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1695999999999998e-05 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488851+0000", 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102168 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8254 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71958547700001, 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020006448999999999, 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.210 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8254 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469008+0000", 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469007+0000", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469010+0000", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469023+0000", 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3747000000000001e-05 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469029+0000", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5010000000000004e-06 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488886+0000", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019857443999999998 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.488956+0000", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9918000000000004e-05 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489014+0000", 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7992000000000003e-05 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8255 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71955972800001, 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020156553000000001, 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.214 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8255 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469034+0000", 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469033+0000", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469036+0000", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5490000000000001e-06 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469042+0000", 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0549999999999996e-06 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469048+0000", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489044+0000", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019995902999999999 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489118+0000", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4176000000000003e-05 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489190+0000", 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1907000000000001e-05 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8256 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.719538699, 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020266409999999999, 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.218 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8256 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469055+0000", 2024-09-21T14:23:46.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469054+0000", 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469056+0000", 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.497e-06 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469070+0000", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3062e-05 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469076+0000", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3389999999999998e-06 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489220+0000", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020143807999999999 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489263+0000", 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3720999999999997e-05 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489321+0000", 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8257 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:46.222 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71951284900001, 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020354787999999999, 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8257 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469080+0000", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469083+0000", 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469089+0000", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1299999999999998e-06 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469095+0000", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7649999999999999e-06 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489348+0000", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020253182000000002 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489391+0000", 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3384999999999999e-05 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489435+0000", 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4159999999999997e-05 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8258 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71949437000001, 2024-09-21T14:23:46.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020462876000000001, 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8258 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469099+0000", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469098+0000", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469101+0000", 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2620000000000001e-06 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469108+0000", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8179999999999998e-06 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469112+0000", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489470+0000", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020357770000000001 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489519+0000", 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9398999999999999e-05 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489562+0000", 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2614999999999998e-05 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8259 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71947624800001, 2024-09-21T14:23:46.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020550230999999999, 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8259 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469117+0000", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469119+0000", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3269999999999999e-06 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469126+0000", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469131+0000", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489584+0000", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020453328 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489626+0000", 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2134e-05 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489667+0000", 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0911999999999997e-05 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8260 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:46.234 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71945588599999, 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020679698999999999, 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8260 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469137+0000", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469139+0000", 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2979999999999999e-06 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469148+0000", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3240000000000003e-06 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469153+0000", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489705+0000", 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020552527000000001 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489776+0000", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1043000000000004e-05 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489817+0000", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0717999999999999e-05 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8261 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:46.238 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.719436194, 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020769942, 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8261 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469157+0000", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469159+0000", 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469166+0000", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5459999999999997e-06 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469170+0000", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4340000000000002e-06 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489841+0000", 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020671510000000001 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489885+0000", 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3829000000000003e-05 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489927+0000", 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1749000000000003e-05 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8262 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71941921600001, 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020968693, 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8262 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469174+0000", 2024-09-21T14:23:46.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469177+0000", 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469184+0000", 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1740000000000003e-06 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469189+0000", 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.489969+0000", 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020779348 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490062+0000", 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3022000000000005e-05 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490143+0000", 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1042000000000001e-05 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8263 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.719399544, 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021129370000000001, 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8263 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469194+0000", 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469193+0000", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469207+0000", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3759e-05 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469214+0000", 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9009999999999997e-06 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469219+0000", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490173+0000", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020953897999999999 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490244+0000", 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1432000000000003e-05 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490323+0000", 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9050999999999999e-05 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8264 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71937008099999, 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021288241999999999, 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.252 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8264 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469223+0000", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469225+0000", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4380000000000002e-06 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469231+0000", 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8459999999999996e-06 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469238+0000", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.092e-06 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490361+0000", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021122297000000002 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490440+0000", 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9086999999999997e-05 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490511+0000", 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1863e-05 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8265 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.719327835, 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021382858000000001, 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8265 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469265+0000", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469268+0000", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8329999999999998e-06 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469275+0000", 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3220000000000004e-06 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469280+0000", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1270000000000002e-06 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490551+0000", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021270415000000001 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490601+0000", 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0584000000000002e-05 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490648+0000", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7120000000000003e-05 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8266 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71930884700001, 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021535892000000001, 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.260 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8266 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469284+0000", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469291+0000", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-06 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469297+0000", 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469303+0000", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490683+0000", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021379390000000002 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490754+0000", 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1789e-05 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490820+0000", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6042000000000002e-05 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8267 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.71928548899999, 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021622410000000002, 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.264 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8267 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469308+0000", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469307+0000", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469310+0000", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.368e-06 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469321+0000", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1429e-05 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469327+0000", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490847+0000", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021520042 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490889+0000", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1680000000000001e-05 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490930+0000", 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1692e-05 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:8268 3.5 3:a18b8141:::rbd_data.11214feba615.0000000000000062:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 136.719262136, 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021708222999999999, 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 8268 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469331+0000", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469333+0000", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2840000000000001e-06 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469339+0000", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.469343+0000", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3880000000000002e-06 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490949+0000", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021605804999999999 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.490999+0000", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9894e-05 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:29.491039+0000", 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0333000000000002e-05 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.272 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:46.273 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:23:46.274 INFO:teuthology.orchestra.run.smithi073.stdout:All operations completed A-OK! 2024-09-21T14:23:46.274 DEBUG:teuthology.parallel:result is None 2024-09-21T14:23:46.274 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T14:23:46.283 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T14:23:46.291 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T14:23:46.292 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-09-21T14:23:46.273+0000 7fc340c39640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33047) UID: 0 2024-09-21T14:23:46.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:23:46.420 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:46.420 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:46.420 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-21T14:23:46.421 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:46.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:23:46.576 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:46.576 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-21T14:23:46.576 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-21T14:23:46.576 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-21T14:23:46.576 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:46.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:23:46.690 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:23:46.690 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-21T14:23:46.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-21T14:23:46.690 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-21T14:23:46.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4172.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 157.148019893, 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19798147499999999, 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4172", 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/3582297441", 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541939+0000", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541938+0000", 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541946+0000", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8899999999999996e-06 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541957+0000", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0876000000000001e-05 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.541965+0000", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.481e-06 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542031+0000", 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6381e-05 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.542040+0000", 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738399+0000", 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196358958 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738608+0000", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020936699999999999 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.738819+0000", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000210707 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739504+0000", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068573400000000002 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739847+0000", 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034280600000000002 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739886+0000", 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8349000000000002e-05 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:09.739921+0000", 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5213999999999999e-05 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43098 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302429499, 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040401099000000003, 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43098 2024-09-21T14:23:46.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387530+0000", 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387529+0000", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387531+0000", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387537+0000", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.31e-06 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387542+0000", 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4829999999999999e-06 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427924+0000", 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040381663999999998 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427931+0000", 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.956e-06 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43099 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.3024135, 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040412579999999997, 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43099 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387546+0000", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387545+0000", 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387547+0000", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.821e-06 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387552+0000", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1499999999999998e-06 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387557+0000", 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2690000000000001e-06 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427952+0000", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040394869 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427958+0000", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7909999999999999e-06 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43100 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:46.704 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302398604, 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040429344999999998, 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43100 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387561+0000", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387560+0000", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387562+0000", 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9130000000000001e-06 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387568+0000", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4790000000000004e-06 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387572+0000", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2289999999999997e-06 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427979+0000", 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040406769000000002 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.427990+0000", 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43101 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302383668, 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040442978999999997, 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43101 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387576+0000", 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387575+0000", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387577+0000", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8729999999999999e-06 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387583+0000", 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6049999999999997e-06 2024-09-21T14:23:46.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387587+0000", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2409999999999997e-06 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428012+0000", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040424760999999997 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428019+0000", 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8190000000000003e-06 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43102 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302368688, 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040455818999999997, 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43102 2024-09-21T14:23:46.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387591+0000", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387590+0000", 2024-09-21T14:23:46.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387592+0000", 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.773e-06 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387598+0000", 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000003e-06 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387602+0000", 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1389999999999997e-06 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428039+0000", 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040437689999999998 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428046+0000", 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9909999999999996e-06 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43103 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302354072, 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040469487999999998, 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43103 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387605+0000", 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387607+0000", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.88e-06 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387612+0000", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6160000000000001e-06 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387617+0000", 2024-09-21T14:23:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1899999999999997e-06 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428068+0000", 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040451262000000002 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428075+0000", 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8589999999999999e-06 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43104 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.30233902499999, 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040482720999999999, 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43104 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387620+0000", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387622+0000", 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8199999999999999e-06 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387627+0000", 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4809999999999997e-06 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387631+0000", 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8519999999999997e-06 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428096+0000", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040465068 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428103+0000", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-06 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43105 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:46.723 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302324466, 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040495626999999999, 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43105 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387635+0000", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387634+0000", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387636+0000", 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9420000000000002e-06 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387642+0000", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.502e-06 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387646+0000", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9210000000000002e-06 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428124+0000", 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040478083999999998 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428130+0000", 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.516e-06 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43106 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302309741, 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040559886000000003, 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.727 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43106 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387649+0000", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387651+0000", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.903e-06 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387656+0000", 2024-09-21T14:23:46.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3689999999999998e-06 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387660+0000", 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0570000000000003e-06 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428202+0000", 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040541621999999999 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428209+0000", 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2880000000000004e-06 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43107 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302295301, 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040614983, 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43107 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387664+0000", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387665+0000", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.827e-06 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387671+0000", 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2730000000000002e-06 2024-09-21T14:23:46.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387675+0000", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9709999999999998e-06 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428271+0000", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040596523000000002 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428279+0000", 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6699999999999994e-06 2024-09-21T14:23:46.734 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43108 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302280951, 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040630237, 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.735 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43108 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387678+0000", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387684+0000", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5080000000000002e-06 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387690+0000", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7529999999999998e-06 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387695+0000", 2024-09-21T14:23:46.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428301+0000", 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040606827999999998 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428309+0000", 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1590000000000004e-06 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43109 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.302260885, 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040639205999999997, 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43109 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387698+0000", 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387700+0000", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9589999999999998e-06 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387705+0000", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4399999999999996e-06 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387710+0000", 2024-09-21T14:23:46.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1760000000000003e-06 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428331+0000", 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040621182999999998 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428338+0000", 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7989999999999996e-06 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43110 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.30224597500001, 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652496000000003, 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43110 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387713+0000", 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387715+0000", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.922e-06 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387720+0000", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387724+0000", 2024-09-21T14:23:46.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.121e-06 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428359+0000", 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040634372000000002 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428366+0000", 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9110000000000004e-06 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43111 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.30223103500001, 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040666474000000001, 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43111 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387728+0000", 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387730+0000", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8390000000000001e-06 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387740+0000", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0265000000000001e-05 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387744+0000", 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2359999999999998e-06 2024-09-21T14:23:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428387+0000", 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040643222 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428395+0000", 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2439999999999996e-06 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:43112 3.1d 3.c5d7469d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 119.30221134200001, 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040672445000000002, 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 43112 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387748+0000", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387749+0000", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387756+0000", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9879999999999998e-06 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.387760+0000", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0219999999999998e-06 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428413+0000", 2024-09-21T14:23:46.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.040652604000000002 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:21:47.428420+0000", 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2529999999999998e-06 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4385.0:80979 3.1e e77/47 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head v 77'229, mlcod=72'227)", 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 82.654730452999999, 2024-09-21T14:23:46.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053494221000000002, 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035229+0000", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.035228+0000", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066600+0000", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031371588999999998 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066625+0000", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5473000000000002e-05 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066635+0000", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3090000000000001e-06 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066721+0000", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6044000000000005e-05 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.066748+0000", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7498999999999999e-05 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:46.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088676+0000", 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021927631 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.088723+0000", 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7233e-05 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:81237 3.9 3:930890d3:::rbd_data.1121590fed9b.0000000000000001:head [call rbd.sparse_copyup in=29733940b,set-alloc-hint object_size 33554432 write_size 33554432,write 5447720~15783 in=15783b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:46.758 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 82.544581269999995, 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.058324474000000001, 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 81237 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145378+0000", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.145376+0000", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153101+0000", 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0077251430000000003 2024-09-21T14:23:46.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153128+0000", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7015000000000001e-05 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153137+0000", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1800000000000002e-06 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153193+0000", 2024-09-21T14:23:46.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5893000000000002e-05 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.153352+0000", 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015924999999999999 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.154485+0000", 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001133121 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:46.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.185530+0000", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.031045116000000001 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203631+0000", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018100491 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203669+0000", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7815000000000003e-05 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:24.203702+0000", 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3759e-05 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:82756 3.9 3:960232d9:::rbd_data.1121a2b38c70.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 81.641018725999999, 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.047683804000000003, 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 82756 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048941+0000", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.048938+0000", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058507+0000", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095690310000000004 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058527+0000", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9561000000000001e-05 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058535+0000", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6889999999999993e-06 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058589+0000", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3637000000000002e-05 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.058704+0000", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115271 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:46.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.060684+0000", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001979743 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.082021+0000", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021337213000000001 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096485+0000", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014464098999999999 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096577+0000", 2024-09-21T14:23:46.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2074e-05 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:22:25.096624+0000", 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6848999999999998e-05 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4385.0:99795 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.43285541, 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91399163500000002, 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-21T14:23:46.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4385", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99795 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-21T14:23:46.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257104+0000", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257103+0000", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257108+0000", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.374e-06 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-21T14:23:46.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257116+0000", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2900000000000002e-06 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257120+0000", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5709999999999999e-06 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257160+0000", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9406000000000002e-05 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-21T14:23:46.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:33.257167+0000", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.781e-06 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162276+0000", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.905108939 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162390+0000", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114109 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7", 2024-09-21T14:23:46.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.162524+0000", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013418600000000001 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.163436+0000", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091262600000000004 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.170942+0000", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0075057079999999998 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-21T14:23:46.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.171067+0000", 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124475 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-21T14:23:34.171095+0000", 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8839999999999998e-05 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-21T14:23:46.775 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:23:46.776 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:23:46.890 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:46.890 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:46.890 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:46.890 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:46.899 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:23:47.001 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.001 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:47.001 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:47.001 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:47.001 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:23:47.112 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.112 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78855 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.345156911000004, 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75346897300000004, 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78855 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.113 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766980+0000", 2024-09-21T14:23:47.114 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766984+0000", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9639999999999997e-06 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766992+0000", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8540000000000003e-06 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.766999+0000", 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6359999999999997e-06 2024-09-21T14:23:47.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767036+0000", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7055999999999999e-05 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767043+0000", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8299999999999998e-06 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520438+0000", 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75339522999999997 2024-09-21T14:23:47.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520449+0000", 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1004e-05 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78858 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.345080401000004, 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75343903000000001, 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.117 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78858 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767057+0000", 2024-09-21T14:23:47.118 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767056+0000", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767060+0000", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4879999999999999e-06 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767068+0000", 2024-09-21T14:23:47.119 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1149999999999994e-06 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767073+0000", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767128+0000", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4500999999999999e-05 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767135+0000", 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.418e-06 2024-09-21T14:23:47.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520486+0000", 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75335110400000005 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520496+0000", 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2220000000000008e-06 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78881 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.121 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.344270437000006, 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75269632399999997, 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78881 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767867+0000", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767866+0000", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767870+0000", 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7950000000000001e-06 2024-09-21T14:23:47.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767878+0000", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3059999999999998e-06 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767884+0000", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9739999999999999e-06 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767929+0000", 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5018000000000002e-05 2024-09-21T14:23:47.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767936+0000", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8650000000000003e-06 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520549+0000", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75261322100000005 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520563+0000", 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3738e-05 2024-09-21T14:23:47.125 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78882 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.344243379999995, 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75268714999999997, 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.126 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78882 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767894+0000", 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.127 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767893+0000", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.767896+0000", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.604e-06 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768323+0000", 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042759100000000001 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.128 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768330+0000", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9310000000000002e-06 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768370+0000", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9368e-05 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768377+0000", 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1980000000000004e-06 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520575+0000", 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75219827299999997 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520581+0000", 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6250000000000004e-06 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.130 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78883 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343795157000002, 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75228544500000005, 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78883 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.131 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768342+0000", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768341+0000", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.132 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768345+0000", 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8809999999999998e-06 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768354+0000", 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4489999999999999e-06 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768359+0000", 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2850000000000002e-06 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768408+0000", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9193000000000001e-05 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768415+0000", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.126e-06 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520617+0000", 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75220142800000001 2024-09-21T14:23:47.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520627+0000", 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0730999999999999e-05 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78889 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343772817000001, 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230459100000002, 2024-09-21T14:23:47.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78889 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768364+0000", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768366+0000", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3860000000000001e-06 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768373+0000", 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5939999999999999e-06 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768378+0000", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0429999999999998e-06 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768440+0000", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2556000000000002e-05 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768446+0000", 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9939999999999997e-06 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520662+0000", 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75221529300000001 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520669+0000", 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78890 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343731125999994, 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75230863599999998, 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78890 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:47.140 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768406+0000", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768405+0000", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768409+0000", 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3900000000000002e-06 2024-09-21T14:23:47.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768417+0000", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9419999999999995e-06 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768423+0000", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8510000000000004e-06 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768468+0000", 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5599999999999997e-05 2024-09-21T14:23:47.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768474+0000", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7679999999999997e-06 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520702+0000", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75222813399999999 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520715+0000", 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2435e-05 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78897 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343572460999994, 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75216472499999998, 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78897 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768565+0000", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768564+0000", 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.145 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768573+0000", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8240000000000006e-06 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768580+0000", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0550000000000002e-06 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768585+0000", 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3489999999999999e-06 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9283999999999999e-05 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768631+0000", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1809999999999998e-06 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520725+0000", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209384000000001 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.147 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520729+0000", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78898 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343540039999993, 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215719999999997, 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.148 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78898 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768597+0000", 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768599+0000", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768606+0000", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.596e-06 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768611+0000", 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8799999999999999e-06 2024-09-21T14:23:47.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768650+0000", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9144000000000003e-05 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768657+0000", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5449999999999998e-06 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520745+0000", 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.752087846 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520754+0000", 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0739999999999999e-06 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78899 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343511511000003, 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75214437499999998, 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.152 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78899 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768626+0000", 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768625+0000", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768628+0000", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.0520000000000002e-06 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768635+0000", 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6229999999999999e-06 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.154 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768640+0000", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0130000000000001e-06 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768676+0000", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5862999999999999e-05 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8039999999999998e-06 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.155 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520765+0000", 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208356799999998 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520770+0000", 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8720000000000001e-06 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78905 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343490415000005, 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75215151999999996, 2024-09-21T14:23:47.156 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78905 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768647+0000", 2024-09-21T14:23:47.157 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768646+0000", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768649+0000", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.305e-06 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768656+0000", 2024-09-21T14:23:47.158 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9659999999999999e-06 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768660+0000", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8860000000000003e-06 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768699+0000", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9001999999999997e-05 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768706+0000", 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:23:47.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520789+0000", 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208317899999999 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520798+0000", 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.3989999999999993e-06 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78906 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.160 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343458244999994, 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75213474999999996, 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78906 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.161 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768679+0000", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768678+0000", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768681+0000", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6869999999999999e-06 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.162 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768688+0000", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0609999999999998e-06 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768693+0000", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768723+0000", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9811e-05 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768728+0000", 2024-09-21T14:23:47.163 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0950000000000003e-06 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520809+0000", 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208024100000004 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520814+0000", 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0250000000000002e-06 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.164 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78913 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343381755999999, 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209215100000004, 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78913 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:47.165 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768755+0000", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768757+0000", 2024-09-21T14:23:47.166 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4159999999999998e-06 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768764+0000", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1940000000000003e-06 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768771+0000", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768801+0000", 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9779000000000001e-05 2024-09-21T14:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768810+0000", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.4730000000000006e-06 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520837+0000", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75202700300000003 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520847+0000", 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0166999999999999e-05 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.168 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78914 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343358816000006, 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75208666899999999, 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78914 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.169 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768778+0000", 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.170 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768780+0000", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4849999999999999e-06 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768786+0000", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2790000000000004e-06 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768791+0000", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3109999999999999e-06 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.171 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768853+0000", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1921000000000006e-05 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768860+0000", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2840000000000001e-06 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520861+0000", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75200058299999994 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520865+0000", 2024-09-21T14:23:47.172 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3390000000000003e-06 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78915 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343341799000001, 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75209453199999998, 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78915 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.173 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768795+0000", 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768797+0000", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768803+0000", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9240000000000004e-06 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768807+0000", 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3669999999999999e-06 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769638+0000", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00083107299999999999 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769646+0000", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4159999999999998e-06 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520881+0000", 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75123469799999998 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.176 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520890+0000", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2110000000000004e-06 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78929 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343192946000002, 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75196196500000001, 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.177 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78929 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.178 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768944+0000", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768946+0000", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1129999999999999e-06 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768951+0000", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6450000000000002e-06 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.179 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768956+0000", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3800000000000004e-06 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769726+0000", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00076976699999999998 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769732+0000", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4829999999999996e-06 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520902+0000", 2024-09-21T14:23:47.180 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75116980899999997 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520906+0000", 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1400000000000002e-06 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78930 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343177236000002, 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75197086800000001, 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.181 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78930 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:47.182 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768960+0000", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768962+0000", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768967+0000", 2024-09-21T14:23:47.183 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768971+0000", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2100000000000003e-06 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769753+0000", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000781819 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769758+0000", 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.242e-06 2024-09-21T14:23:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520922+0000", 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751163197 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520931+0000", 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78933 3.e 3.5a81514e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.185 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343161812999995, 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.751971273, 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78933 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768975+0000", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768977+0000", 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.187 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768983+0000", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7069999999999997e-06 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.768987+0000", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4360000000000004e-06 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769778+0000", 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00079070099999999999 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.188 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769783+0000", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1089999999999997e-06 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520940+0000", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75115702900000003 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.520947+0000", 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4069999999999998e-06 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78954 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.343029748000006, 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192138600000002, 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78954 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.190 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769107+0000", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.191 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769109+0000", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1619999999999998e-06 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769117+0000", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7810000000000005e-06 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769123+0000", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6370000000000004e-06 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.192 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770163+0000", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001039994 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770168+0000", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2000000000000002e-06 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521020+0000", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75085217100000001 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521029+0000", 2024-09-21T14:23:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7679999999999997e-06 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:78956 3.e 3.f066560e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 87.342994691000001, 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75192767800000004, 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "reached pg", 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 78956 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.195 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769142+0000", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769144+0000", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0600000000000002e-06 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769149+0000", 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3669999999999996e-06 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.196 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.769155+0000", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2329999999999997e-06 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770211+0000", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00105619 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:19.770216+0000", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1200000000000001e-06 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521061+0000", 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.75084535600000002 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:20.521070+0000", 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6969999999999999e-06 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.198 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:23:47.301 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.301 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:47.301 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:47.301 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:23:47.404 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T14:23:47.404 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-21T14:23:47.413 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.413 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:47.414 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:47.414 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:47.414 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:23:47.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:23:47.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:23:47.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:23:47.526 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:80979 3.1e 3:7de22753:::rbd_data.1121590fed9b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1179486~17832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.524015872999996, 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.087635822000000002, 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:47.527 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 80979 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.528 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001610+0000", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.001609+0000", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031112+0000", 2024-09-21T14:23:47.529 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.029502761999999998 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031157+0000", 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4672000000000002e-05 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031182+0000", 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.5748000000000001e-05 2024-09-21T14:23:47.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031243+0000", 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1074999999999994e-05 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.031402+0000", 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000158622 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 3", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.032655+0000", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0012527809999999999 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.064945+0000", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.032290118 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089091+0000", 2024-09-21T14:23:47.532 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.024146056999999999 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089182+0000", 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1308000000000001e-05 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:24.089246+0000", 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3503000000000005e-05 2024-09-21T14:23:47.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84762 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.027396053000004, 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071865522000000001, 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.534 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84762 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.535 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498230+0000", 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498229+0000", 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498234+0000", 2024-09-21T14:23:47.536 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.887e-06 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498242+0000", 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3000000000000002e-06 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498249+0000", 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8469999999999998e-06 2024-09-21T14:23:47.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567747+0000", 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.069497430999999998 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.567839+0000", 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.2788999999999995e-05 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.538 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570095+0000", 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0022560850000000001 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84763 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24379392~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.027370482999999, 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072389820999999993, 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84763 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.540 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498255+0000", 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498257+0000", 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.4700000000000001e-06 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498264+0000", 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5220000000000004e-06 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.542 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.498273+0000", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0399999999999998e-06 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570117+0000", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071844247 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570206+0000", 2024-09-21T14:23:47.543 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.8749e-05 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570645+0000", 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043930599999999998 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84770 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19398656~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.544 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.026416095000002, 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071954313000000006, 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84770 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499210+0000", 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499209+0000", 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499213+0000", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2209999999999999e-06 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499221+0000", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3780000000000003e-06 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499224+0000", 2024-09-21T14:23:47.547 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570672+0000", 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071447572000000001 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.570731+0000", 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9431000000000001e-05 2024-09-21T14:23:47.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571164+0000", 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00043279800000000003 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84771 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20447232~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.549 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.026101593999996, 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072197150000000002, 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84771 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.550 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499524+0000", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499523+0000", 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499529+0000", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4469999999999997e-06 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499541+0000", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1693e-05 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.499549+0000", 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5150000000000006e-06 2024-09-21T14:23:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571189+0000", 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071640337999999998 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571269+0000", 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0116000000000003e-05 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.553 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.571721+0000", 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045197599999999998 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84779 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.024686340000002, 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071992475, 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.554 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84779 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500940+0000", 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500939+0000", 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.556 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500944+0000", 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3380000000000004e-06 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500954+0000", 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0200999999999999e-05 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.500961+0000", 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.0119999999999999e-06 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572385+0000", 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071423722999999995 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572465+0000", 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0490999999999999e-05 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.572932+0000", 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00046667699999999998 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84787 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 14417920~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.023638890000001, 2024-09-21T14:23:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072168974999999996, 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84787 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.560 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501987+0000", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501986+0000", 2024-09-21T14:23:47.561 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501990+0000", 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1659999999999996e-06 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.501999+0000", 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4789999999999996e-06 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.502005+0000", 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3439999999999997e-06 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573552+0000", 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071546790999999998 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.563 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.573640+0000", 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7709999999999999e-05 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574156+0000", 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00051617299999999998 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.564 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84795 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 10223616~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.022355476000001, 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072175960999999997, 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84795 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.565 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.566 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503270+0000", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503274+0000", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8840000000000004e-06 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503282+0000", 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8660000000000006e-06 2024-09-21T14:23:47.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.503287+0000", 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6869999999999999e-06 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574858+0000", 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071570588000000004 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.568 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.574913+0000", 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5587999999999997e-05 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575446+0000", 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00053301399999999995 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84803 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 13893632~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.02103425, 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071915106000000006, 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84803 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504592+0000", 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.571 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504591+0000", 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504594+0000", 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1209999999999998e-06 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504603+0000", 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6419999999999996e-06 2024-09-21T14:23:47.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.504609+0000", 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.575983+0000", 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071373865999999994 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.573 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576036+0000", 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3504000000000001e-05 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576507+0000", 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000470438 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.574 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84810 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 8650752~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.019993315999997, 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073967206999999993, 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84810 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505633+0000", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505632+0000", 2024-09-21T14:23:47.576 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505637+0000", 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6890000000000003e-06 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505645+0000", 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0160000000000008e-06 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.577 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505649+0000", 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6360000000000002e-06 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576556+0000", 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.070906845999999996 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.576662+0000", 2024-09-21T14:23:47.578 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000105726 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579600+0000", 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0029380460000000001 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84811 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 9699328~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:47.579 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.019741875999998, 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.074314011999999999, 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84811 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.580 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505884+0000", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505883+0000", 2024-09-21T14:23:47.581 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505888+0000", 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5009999999999998e-06 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505917+0000", 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9067000000000001e-05 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.582 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.505923+0000", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5400000000000001e-06 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579622+0000", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073698448999999999 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.579698+0000", 2024-09-21T14:23:47.583 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.6415999999999995e-05 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580198+0000", 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00049982 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84818 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 29360128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.018696126999998, 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073594788999999994, 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84818 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.585 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506930+0000", 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506929+0000", 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506932+0000", 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.768e-06 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506939+0000", 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6100000000000002e-06 2024-09-21T14:23:47.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.506944+0000", 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7810000000000002e-06 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580304+0000", 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073359416999999996 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.588 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580435+0000", 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000130689 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580525+0000", 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.0002999999999999e-05 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84819 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.589 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.018309185000007, 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073419569000000004, 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84819 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.590 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507317+0000", 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507316+0000", 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507321+0000", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6160000000000004e-06 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507332+0000", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1861999999999999e-05 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.507339+0000", 2024-09-21T14:23:47.592 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4509999999999997e-06 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580565+0000", 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073226581999999998 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580649+0000", 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3997000000000002e-05 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580736+0000", 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6806999999999997e-05 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84826 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 23068672~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.017373601000003, 2024-09-21T14:23:47.594 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073045657, 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84826 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508252+0000", 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.596 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508256+0000", 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2570000000000001e-06 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508269+0000", 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2883e-05 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508275+0000", 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9499999999999998e-06 2024-09-21T14:23:47.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580783+0000", 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072508638 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.580845+0000", 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1948999999999999e-05 2024-09-21T14:23:47.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581298+0000", 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00045263999999999998 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84827 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 24117248~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.599 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.017280932000006, 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.073465805999999995, 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84827 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.600 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508345+0000", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508344+0000", 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508348+0000", 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1420000000000003e-06 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508357+0000", 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1699999999999997e-06 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.508363+0000", 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0179999999999998e-06 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581339+0000", 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072976813000000001 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.603 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581407+0000", 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7492999999999999e-05 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.581811+0000", 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00040385399999999999 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84854 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31195136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.013199713999995, 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071767585999999994, 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84854 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.605 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.606 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512426+0000", 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512429+0000", 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7409999999999998e-06 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.607 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512437+0000", 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5460000000000003e-06 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512443+0000", 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2620000000000004e-06 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584037+0000", 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071593931999999999 2024-09-21T14:23:47.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584108+0000", 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1123000000000003e-05 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584194+0000", 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5638999999999998e-05 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84855 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 30408704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.013016882000002, 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071795933000000006, 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.610 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84855 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512609+0000", 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512608+0000", 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512614+0000", 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4960000000000004e-06 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512625+0000", 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1321999999999999e-05 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512632+0000", 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4889999999999998e-06 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.613 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584233+0000", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071601068000000004 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584337+0000", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000103016 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584405+0000", 2024-09-21T14:23:47.614 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8454000000000006e-05 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84857 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 31457280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.012693170000006, 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071918438000000001, 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.615 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84857 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.616 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512933+0000", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512932+0000", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512935+0000", 2024-09-21T14:23:47.617 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8959999999999999e-06 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512942+0000", 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4910000000000002e-06 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.512946+0000", 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000004e-06 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584675+0000", 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071728832000000006 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584750+0000", 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4257000000000004e-05 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.619 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584851+0000", 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000101609 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84858 3.0 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:2e [sparse-read 33292288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.012571717, 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071908957999999995, 2024-09-21T14:23:47.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84858 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513054+0000", 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513053+0000", 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513057+0000", 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5619999999999999e-06 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513067+0000", 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0356999999999999e-05 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.623 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.513073+0000", 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6529999999999999e-06 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584870+0000", 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071797436000000006 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.624 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584912+0000", 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1576999999999999e-05 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.584963+0000", 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1138999999999999e-05 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:84867 3.0 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:2d [sparse-read 19922944~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:23:47.625 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 74.011313118000004, 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.072202529000000001, 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 84867 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.626 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514313+0000", 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514312+0000", 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514315+0000", 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.39e-06 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514321+0000", 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9100000000000002e-06 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.628 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.514325+0000", 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586069+0000", 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.071744422000000002 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586142+0000", 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3038000000000001e-05 2024-09-21T14:23:47.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:33.586515+0000", 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00037289099999999998 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.630 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:23:47.735 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.735 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:47.735 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:47.735 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:23:47.747 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:23:47.748 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:23:47.759 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-21T14:23:47.403148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107055327,"num_objects":481,"num_object_clones":34,"num_object_copies":962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":481,"num_whiteouts":414,"num_read":148362,"num_read_kb":954268,"num_write":21870,"num_write_kb":506140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19330,"ondisk_log_size":19330,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":357044,"kb_used_data":14532,"kb_used_omap":10,"kb_used_meta":342389,"kb_avail":660245836,"statfs":{"total":676457349120,"available":676091736064,"internally_reserved":0,"allocated":14880768,"data_stored":10280779,"data_compressed":198990,"data_compressed_allocated":413696,"data_compressed_original":2826240,"omap_allocated":11150,"internal_metadata":350606450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.248135"},"pg_stats":[{"pgid":"3.1f","version":"155'393","reported_seq":3662,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.261573+0000","last_change":"2024-09-21T14:23:42.223561+0000","last_active":"2024-09-21T14:23:42.261573+0000","last_peered":"2024-09-21T14:23:42.261573+0000","last_clean":"2024-09-21T14:23:42.261573+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:23:42.261573+0000","last_undegraded":"2024-09-21T14:23:42.261573+0000","last_fullsized":"2024-09-21T14:23:42.261573+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090032000000000003,"stat_sum":{"num_bytes":167675,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3820,"num_read_kb":17765,"num_write":359,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1e","version":"154'342","reported_seq":2484,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.222795+0000","last_change":"2024-09-21T14:23:42.223174+0000","last_active":"2024-09-21T14:23:42.222795+0000","last_peered":"2024-09-21T14:23:42.222795+0000","last_clean":"2024-09-21T14:23:42.222795+0000","last_became_active":"2024-09-21T14:23:33.163881+0000","last_became_peered":"2024-09-21T14:23:33.163881+0000","last_unstale":"2024-09-21T14:23:42.222795+0000","last_undegraded":"2024-09-21T14:23:42.222795+0000","last_fullsized":"2024-09-21T14:23:42.222795+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:57:22.066984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053884199999999999,"stat_sum":{"num_bytes":3207094,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":321,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1d","version":"155'449","reported_seq":3513,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258425+0000","last_change":"2024-09-21T14:23:42.161506+0000","last_active":"2024-09-21T14:23:42.258425+0000","last_peered":"2024-09-21T14:23:42.258425+0000","last_clean":"2024-09-21T14:23:42.258425+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:23:42.258425+0000","last_undegraded":"2024-09-21T14:23:42.258425+0000","last_fullsized":"2024-09-21T14:23:42.258425+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015082170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1c","version":"155'513","reported_seq":6296,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.261642+0000","last_change":"2024-09-21T14:23:42.171608+0000","last_active":"2024-09-21T14:23:42.261642+0000","last_peered":"2024-09-21T14:23:42.261642+0000","last_clean":"2024-09-21T14:23:42.261642+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:23:42.261642+0000","last_undegraded":"2024-09-21T14:23:42.261642+0000","last_fullsized":"2024-09-21T14:23:42.261642+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064302000000000001,"stat_sum":{"num_bytes":3995317,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":6528,"num_read_kb":83143,"num_write":550,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1b","version":"154'565","reported_seq":4967,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155462+0000","last_change":"2024-09-21T14:23:42.155799+0000","last_active":"2024-09-21T14:23:42.155462+0000","last_peered":"2024-09-21T14:23:42.155462+0000","last_clean":"2024-09-21T14:23:42.155462+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:23:42.155462+0000","last_undegraded":"2024-09-21T14:23:42.155462+0000","last_fullsized":"2024-09-21T14:23:42.155462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075175700000000003,"stat_sum":{"num_bytes":65536,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1a","version":"154'556","reported_seq":5765,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.301478+0000","last_change":"2024-09-21T14:23:42.171902+0000","last_active":"2024-09-21T14:23:42.301478+0000","last_peered":"2024-09-21T14:23:42.301478+0000","last_clean":"2024-09-21T14:23:42.301478+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:23:42.301478+0000","last_undegraded":"2024-09-21T14:23:42.301478+0000","last_fullsized":"2024-09-21T14:23:42.301478+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051669800000000001,"stat_sum":{"num_bytes":485403,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":13,"num_read":7186,"num_read_kb":66194,"num_write":578,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.19","version":"153'818","reported_seq":6126,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155915+0000","last_change":"2024-09-21T14:23:42.155915+0000","last_active":"2024-09-21T14:23:42.155915+0000","last_peered":"2024-09-21T14:23:42.155915+0000","last_clean":"2024-09-21T14:23:42.155915+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:23:42.155915+0000","last_undegraded":"2024-09-21T14:23:42.155915+0000","last_fullsized":"2024-09-21T14:23:42.155915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081868900000000005,"stat_sum":{"num_bytes":2142032,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":7452,"num_read_kb":30274,"num_write":1071,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.18","version":"155'296","reported_seq":2112,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.257112+0000","last_change":"2024-09-21T14:23:42.223942+0000","last_active":"2024-09-21T14:23:42.257112+0000","last_peered":"2024-09-21T14:23:42.257112+0000","last_clean":"2024-09-21T14:23:42.257112+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:23:42.257112+0000","last_undegraded":"2024-09-21T14:23:42.257112+0000","last_fullsized":"2024-09-21T14:23:42.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242579999999999,"stat_sum":{"num_bytes":471028,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.17","version":"155'833","reported_seq":3692,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258845+0000","last_change":"2024-09-21T14:23:42.223750+0000","last_active":"2024-09-21T14:23:42.258845+0000","last_peered":"2024-09-21T14:23:42.258845+0000","last_clean":"2024-09-21T14:23:42.258845+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:23:42.258845+0000","last_undegraded":"2024-09-21T14:23:42.258845+0000","last_fullsized":"2024-09-21T14:23:42.258845+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000884653,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.16","version":"155'529","reported_seq":2931,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.161759+0000","last_change":"2024-09-21T14:23:42.161759+0000","last_active":"2024-09-21T14:23:42.161759+0000","last_peered":"2024-09-21T14:23:42.161759+0000","last_clean":"2024-09-21T14:23:42.161759+0000","last_became_active":"2024-09-21T14:23:33.164931+0000","last_became_peered":"2024-09-21T14:23:33.164931+0000","last_unstale":"2024-09-21T14:23:42.161759+0000","last_undegraded":"2024-09-21T14:23:42.161759+0000","last_fullsized":"2024-09-21T14:23:42.161759+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:12.329268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001572101,"stat_sum":{"num_bytes":229376,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.15","version":"155'825","reported_seq":3063,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.203157+0000","last_change":"2024-09-21T14:23:42.203157+0000","last_active":"2024-09-21T14:23:42.203157+0000","last_peered":"2024-09-21T14:23:42.203157+0000","last_clean":"2024-09-21T14:23:42.203157+0000","last_became_active":"2024-09-21T14:23:24.295446+0000","last_became_peered":"2024-09-21T14:23:24.295446+0000","last_unstale":"2024-09-21T14:23:42.203157+0000","last_undegraded":"2024-09-21T14:23:42.203157+0000","last_fullsized":"2024-09-21T14:23:42.203157+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:09:36.195502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011680130000000001,"stat_sum":{"num_bytes":3336042,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.14","version":"155'514","reported_seq":3612,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.256136+0000","last_change":"2024-09-21T14:23:42.160467+0000","last_active":"2024-09-21T14:23:42.256136+0000","last_peered":"2024-09-21T14:23:42.256136+0000","last_clean":"2024-09-21T14:23:42.256136+0000","last_became_active":"2024-09-21T14:23:33.163697+0000","last_became_peered":"2024-09-21T14:23:33.163697+0000","last_unstale":"2024-09-21T14:23:42.256136+0000","last_undegraded":"2024-09-21T14:23:42.256136+0000","last_fullsized":"2024-09-21T14:23:42.256136+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:37:13.347634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058776599999999998,"stat_sum":{"num_bytes":163840,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154753+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:23:41.154753+0000","last_peered":"2024-09-21T14:23:41.154753+0000","last_clean":"2024-09-21T14:23:41.154753+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:23:41.154753+0000","last_undegraded":"2024-09-21T14:23:41.154753+0000","last_fullsized":"2024-09-21T14:23:41.154753+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'451","reported_seq":3170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.287665+0000","last_change":"2024-09-21T14:23:42.203455+0000","last_active":"2024-09-21T14:23:42.287665+0000","last_peered":"2024-09-21T14:23:42.287665+0000","last_clean":"2024-09-21T14:23:42.287665+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:23:42.287665+0000","last_undegraded":"2024-09-21T14:23:42.287665+0000","last_fullsized":"2024-09-21T14:23:42.287665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014078859999999999,"stat_sum":{"num_bytes":65552,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":17,"num_read":3583,"num_read_kb":12922,"num_write":409,"num_write_kb":5718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.172190+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:23:41.172190+0000","last_peered":"2024-09-21T14:23:41.172190+0000","last_clean":"2024-09-21T14:23:41.172190+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:23:41.172190+0000","last_undegraded":"2024-09-21T14:23:41.172190+0000","last_fullsized":"2024-09-21T14:23:41.172190+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"155'550","reported_seq":3398,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.202318+0000","last_change":"2024-09-21T14:23:42.202318+0000","last_active":"2024-09-21T14:23:42.202318+0000","last_peered":"2024-09-21T14:23:42.202318+0000","last_clean":"2024-09-21T14:23:42.202318+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:23:42.202318+0000","last_undegraded":"2024-09-21T14:23:42.202318+0000","last_fullsized":"2024-09-21T14:23:42.202318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049983999999999998,"stat_sum":{"num_bytes":2690959,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.157604+0000","last_change":"2024-09-21T14:23:30.327528+0000","last_active":"2024-09-21T14:23:41.157604+0000","last_peered":"2024-09-21T14:23:41.157604+0000","last_clean":"2024-09-21T14:23:41.157604+0000","last_became_active":"2024-09-21T14:23:30.327290+0000","last_became_peered":"2024-09-21T14:23:30.327290+0000","last_unstale":"2024-09-21T14:23:41.157604+0000","last_undegraded":"2024-09-21T14:23:41.157604+0000","last_fullsized":"2024-09-21T14:23:41.157604+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:32.836140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3278,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155404+0000","last_change":"2024-09-21T14:23:42.155404+0000","last_active":"2024-09-21T14:23:42.155404+0000","last_peered":"2024-09-21T14:23:42.155404+0000","last_clean":"2024-09-21T14:23:42.155404+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:23:42.155404+0000","last_undegraded":"2024-09-21T14:23:42.155404+0000","last_fullsized":"2024-09-21T14:23:42.155404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042213500000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.4","version":"0'0","reported_seq":155,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.159947+0000","last_change":"2024-09-21T14:23:24.295710+0000","last_active":"2024-09-21T14:23:41.159947+0000","last_peered":"2024-09-21T14:23:41.159947+0000","last_clean":"2024-09-21T14:23:41.159947+0000","last_became_active":"2024-09-21T14:23:24.295425+0000","last_became_peered":"2024-09-21T14:23:24.295425+0000","last_unstale":"2024-09-21T14:23:41.159947+0000","last_undegraded":"2024-09-21T14:23:41.159947+0000","last_fullsized":"2024-09-21T14:23:41.159947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:55:26.597898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"155'582","reported_seq":5723,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.156110+0000","last_change":"2024-09-21T14:23:42.156110+0000","last_active":"2024-09-21T14:23:42.156110+0000","last_peered":"2024-09-21T14:23:42.156110+0000","last_clean":"2024-09-21T14:23:42.156110+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:23:42.156110+0000","last_undegraded":"2024-09-21T14:23:42.156110+0000","last_fullsized":"2024-09-21T14:23:42.156110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010170120000000001,"stat_sum":{"num_bytes":2572330,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":13,"num_read":6834,"num_read_kb":31516,"num_write":528,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1","version":"155'385","reported_seq":2991,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.249192+0000","last_change":"2024-09-21T14:23:42.202326+0000","last_active":"2024-09-21T14:23:42.249192+0000","last_peered":"2024-09-21T14:23:42.249192+0000","last_clean":"2024-09-21T14:23:42.249192+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:23:42.249192+0000","last_undegraded":"2024-09-21T14:23:42.249192+0000","last_fullsized":"2024-09-21T14:23:42.249192+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051342900000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"2.0","version":"17'117","reported_seq":264,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.171245+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:23:41.171245+0000","last_peered":"2024-09-21T14:23:41.171245+0000","last_clean":"2024-09-21T14:23:41.171245+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:23:41.171245+0000","last_undegraded":"2024-09-21T14:23:41.171245+0000","last_fullsized":"2024-09-21T14:23:41.171245+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":180,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.161134+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:23:41.161134+0000","last_peered":"2024-09-21T14:23:41.161134+0000","last_clean":"2024-09-21T14:23:41.161134+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:23:41.161134+0000","last_undegraded":"2024-09-21T14:23:41.161134+0000","last_fullsized":"2024-09-21T14:23:41.161134+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"154'796","reported_seq":3478,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.223734+0000","last_change":"2024-09-21T14:23:42.224025+0000","last_active":"2024-09-21T14:23:42.223734+0000","last_peered":"2024-09-21T14:23:42.223734+0000","last_clean":"2024-09-21T14:23:42.223734+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:23:42.223734+0000","last_undegraded":"2024-09-21T14:23:42.223734+0000","last_fullsized":"2024-09-21T14:23:42.223734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010560610000000001,"stat_sum":{"num_bytes":16,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":5376,"num_read_kb":124714,"num_write":998,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.2","version":"17'2","reported_seq":42,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.161578+0000","last_change":"2024-09-21T14:23:24.298546+0000","last_active":"2024-09-21T14:23:41.161578+0000","last_peered":"2024-09-21T14:23:41.161578+0000","last_clean":"2024-09-21T14:23:41.161578+0000","last_became_active":"2024-09-21T14:23:24.298163+0000","last_became_peered":"2024-09-21T14:23:24.298163+0000","last_unstale":"2024-09-21T14:23:41.161578+0000","last_undegraded":"2024-09-21T14:23:41.161578+0000","last_fullsized":"2024-09-21T14:23:41.161578+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:52.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"155'1239","reported_seq":2928,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.284764+0000","last_change":"2024-09-21T14:23:42.202788+0000","last_active":"2024-09-21T14:23:42.284764+0000","last_peered":"2024-09-21T14:23:42.284764+0000","last_clean":"2024-09-21T14:23:42.284764+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:23:42.284764+0000","last_undegraded":"2024-09-21T14:23:42.284764+0000","last_fullsized":"2024-09-21T14:23:42.284764+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1239,"log_dups_size":0,"ondisk_log_size":1239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092164299999999998,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":4830,"num_read_kb":16531,"num_write":1688,"num_write_kb":8662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154780+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:23:41.154780+0000","last_peered":"2024-09-21T14:23:41.154780+0000","last_clean":"2024-09-21T14:23:41.154780+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:23:41.154780+0000","last_undegraded":"2024-09-21T14:23:41.154780+0000","last_fullsized":"2024-09-21T14:23:41.154780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"155'605","reported_seq":2780,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.253200+0000","last_change":"2024-09-21T14:23:42.156217+0000","last_active":"2024-09-21T14:23:42.253200+0000","last_peered":"2024-09-21T14:23:42.253200+0000","last_clean":"2024-09-21T14:23:42.253200+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:23:42.253200+0000","last_undegraded":"2024-09-21T14:23:42.253200+0000","last_fullsized":"2024-09-21T14:23:42.253200+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089169399999999995,"stat_sum":{"num_bytes":18190647,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":15,"num_read":3641,"num_read_kb":12437,"num_write":704,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154608+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:23:41.154608+0000","last_peered":"2024-09-21T14:23:41.154608+0000","last_clean":"2024-09-21T14:23:41.154608+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:23:41.154608+0000","last_undegraded":"2024-09-21T14:23:41.154608+0000","last_fullsized":"2024-09-21T14:23:41.154608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"155'666","reported_seq":3718,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.307860+0000","last_change":"2024-09-21T14:23:42.155430+0000","last_active":"2024-09-21T14:23:42.307860+0000","last_peered":"2024-09-21T14:23:42.307860+0000","last_clean":"2024-09-21T14:23:42.307860+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:23:42.307860+0000","last_undegraded":"2024-09-21T14:23:42.307860+0000","last_fullsized":"2024-09-21T14:23:42.307860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056538900000000002,"stat_sum":{"num_bytes":180224,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":5855,"num_read_kb":16010,"num_write":714,"num_write_kb":12338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.9","version":"155'687","reported_seq":4744,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160053+0000","last_change":"2024-09-21T14:23:42.160053+0000","last_active":"2024-09-21T14:23:42.160053+0000","last_peered":"2024-09-21T14:23:42.160053+0000","last_clean":"2024-09-21T14:23:42.160053+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:23:42.160053+0000","last_undegraded":"2024-09-21T14:23:42.160053+0000","last_fullsized":"2024-09-21T14:23:42.160053+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038046199999999998,"stat_sum":{"num_bytes":38988713,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":12,"num_read":7645,"num_read_kb":84617,"num_write":703,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.a","version":"155'467","reported_seq":2298,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160427+0000","last_change":"2024-09-21T14:23:42.160427+0000","last_active":"2024-09-21T14:23:42.160427+0000","last_peered":"2024-09-21T14:23:42.160427+0000","last_clean":"2024-09-21T14:23:42.160427+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:23:42.160427+0000","last_undegraded":"2024-09-21T14:23:42.160427+0000","last_fullsized":"2024-09-21T14:23:42.160427+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057719100000000001,"stat_sum":{"num_bytes":36,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":1310,"num_read_kb":8590,"num_write":431,"num_write_kb":7229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.b","version":"155'474","reported_seq":2532,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.254086+0000","last_change":"2024-09-21T14:23:42.202766+0000","last_active":"2024-09-21T14:23:42.254086+0000","last_peered":"2024-09-21T14:23:42.254086+0000","last_clean":"2024-09-21T14:23:42.254086+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:23:42.254086+0000","last_undegraded":"2024-09-21T14:23:42.254086+0000","last_fullsized":"2024-09-21T14:23:42.254086+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089618900000000004,"stat_sum":{"num_bytes":11025616,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":2728,"num_read_kb":10420,"num_write":567,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.c","version":"155'740","reported_seq":2571,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.157539+0000","last_change":"2024-09-21T14:23:42.157539+0000","last_active":"2024-09-21T14:23:42.157539+0000","last_peered":"2024-09-21T14:23:42.157539+0000","last_clean":"2024-09-21T14:23:42.157539+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:23:42.157539+0000","last_undegraded":"2024-09-21T14:23:42.157539+0000","last_fullsized":"2024-09-21T14:23:42.157539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038724499999999998,"stat_sum":{"num_bytes":2313996,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3998,"num_read_kb":18124,"num_write":914,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.d","version":"155'566","reported_seq":3040,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258670+0000","last_change":"2024-09-21T14:23:42.171832+0000","last_active":"2024-09-21T14:23:42.258670+0000","last_peered":"2024-09-21T14:23:42.258670+0000","last_clean":"2024-09-21T14:23:42.258670+0000","last_became_active":"2024-09-21T14:23:40.370380+0000","last_became_peered":"2024-09-21T14:23:40.370380+0000","last_unstale":"2024-09-21T14:23:42.258670+0000","last_undegraded":"2024-09-21T14:23:42.258670+0000","last_fullsized":"2024-09-21T14:23:42.258670+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:23.244070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066443000000000003,"stat_sum":{"num_bytes":4150664,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.e","version":"155'988","reported_seq":3867,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.305653+0000","last_change":"2024-09-21T14:23:42.223180+0000","last_active":"2024-09-21T14:23:42.305653+0000","last_peered":"2024-09-21T14:23:42.305653+0000","last_clean":"2024-09-21T14:23:42.305653+0000","last_became_active":"2024-09-21T14:23:24.298136+0000","last_became_peered":"2024-09-21T14:23:24.298136+0000","last_unstale":"2024-09-21T14:23:42.305653+0000","last_undegraded":"2024-09-21T14:23:42.305653+0000","last_fullsized":"2024-09-21T14:23:42.305653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:11:03.060692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055806699999999998,"stat_sum":{"num_bytes":167746,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":6087,"num_read_kb":20629,"num_write":1419,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.f","version":"155'489","reported_seq":5024,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.311014+0000","last_change":"2024-09-21T14:23:42.171585+0000","last_active":"2024-09-21T14:23:42.311014+0000","last_peered":"2024-09-21T14:23:42.311014+0000","last_clean":"2024-09-21T14:23:42.311014+0000","last_became_active":"2024-09-21T14:23:39.369052+0000","last_became_peered":"2024-09-21T14:23:39.369052+0000","last_unstale":"2024-09-21T14:23:42.311014+0000","last_undegraded":"2024-09-21T14:23:42.311014+0000","last_fullsized":"2024-09-21T14:23:42.311014+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:29.641991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041964900000000002,"stat_sum":{"num_bytes":1164045,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":16,"num_read":5799,"num_read_kb":25845,"num_write":455,"num_write_kb":21462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.10","version":"154'438","reported_seq":3543,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.171314+0000","last_change":"2024-09-21T14:23:42.171314+0000","last_active":"2024-09-21T14:23:42.171314+0000","last_peered":"2024-09-21T14:23:42.171314+0000","last_clean":"2024-09-21T14:23:42.171314+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:23:42.171314+0000","last_undegraded":"2024-09-21T14:23:42.171314+0000","last_fullsized":"2024-09-21T14:23:42.171314+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040741200000000001,"stat_sum":{"num_bytes":2490368,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.11","version":"155'381","reported_seq":3053,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160439+0000","last_change":"2024-09-21T14:23:42.160439+0000","last_active":"2024-09-21T14:23:42.160439+0000","last_peered":"2024-09-21T14:23:42.160439+0000","last_clean":"2024-09-21T14:23:42.160439+0000","last_became_active":"2024-09-21T14:23:40.369251+0000","last_became_peered":"2024-09-21T14:23:40.369251+0000","last_unstale":"2024-09-21T14:23:42.160439+0000","last_undegraded":"2024-09-21T14:23:42.160439+0000","last_fullsized":"2024-09-21T14:23:42.160439+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:10:23.110922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041244399999999998,"stat_sum":{"num_bytes":4056861,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":18,"num_read":2072,"num_read_kb":23208,"num_write":411,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.12","version":"155'549","reported_seq":3731,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.253300+0000","last_change":"2024-09-21T14:23:42.203183+0000","last_active":"2024-09-21T14:23:42.253300+0000","last_peered":"2024-09-21T14:23:42.253300+0000","last_clean":"2024-09-21T14:23:42.253300+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:23:42.253300+0000","last_undegraded":"2024-09-21T14:23:42.253300+0000","last_fullsized":"2024-09-21T14:23:42.253300+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012320720000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.13","version":"155'1115","reported_seq":4835,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.279864+0000","last_change":"2024-09-21T14:23:42.161499+0000","last_active":"2024-09-21T14:23:42.279864+0000","last_peered":"2024-09-21T14:23:42.279864+0000","last_clean":"2024-09-21T14:23:42.279864+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:23:42.279864+0000","last_undegraded":"2024-09-21T14:23:42.279864+0000","last_fullsized":"2024-09-21T14:23:42.279864+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601119,"stat_sum":{"num_bytes":4143783,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":14,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":106464940,"num_objects":477,"num_object_clones":34,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":414,"num_read":148246,"num_read_kb":954168,"num_write":21709,"num_write_kb":503722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5279,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3334144,"data_stored":17706081,"data_compressed":922618,"data_compressed_allocated":1351680,"data_compressed_original":16193563,"omap_allocated":0,"internal_metadata":0},"log_size":19210,"ondisk_log_size":19210,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2068480,"data_stored":1353789,"data_compressed":31279,"data_compressed_allocated":61440,"data_compressed_original":327680,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.437},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.496},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.375}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2142208,"data_stored":1396311,"data_compressed":22731,"data_compressed_allocated":61440,"data_compressed_original":311296,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.435}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.46800000000000003}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35499999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.37},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55364,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316476,"statfs":{"total":96636764160,"available":96580071424,"internally_reserved":0,"allocated":2150400,"data_stored":1730576,"data_compressed":35148,"data_compressed_allocated":57344,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.32100000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.372},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.36399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.501}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.438}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":2129920,"data_stored":1350001,"data_compressed":25375,"data_compressed_allocated":61440,"data_compressed_original":286720,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.624},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68500000000000005}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69799999999999995}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59484,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312356,"statfs":{"total":96636764160,"available":96575852544,"internally_reserved":0,"allocated":2170880,"data_stored":1898159,"data_compressed":49158,"data_compressed_allocated":86016,"data_compressed_original":823296,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.315},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.502},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72499999999999998}]}]},{"osd":2,"up_from":0,"seq":55834574881,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":2064384,"data_stored":1122455,"data_compressed":4016,"data_compressed_allocated":16384,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.252}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.221},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.31}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.28699999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41299999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.437},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.52400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65224,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":2154496,"data_stored":1429488,"data_compressed":31283,"data_compressed_allocated":69632,"data_compressed_original":364544,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.312}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":237105,"data_compressed":5298,"data_compressed_allocated":24576,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":214058,"data_compressed":4016,"data_compressed_allocated":16384,"data_compressed_original":81920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":281190,"data_compressed":7125,"data_compressed_allocated":24576,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":229970,"data_compressed":4902,"data_compressed_allocated":20480,"data_compressed_original":102400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":204892,"data_compressed":2338,"data_compressed_allocated":8192,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":275748,"data_compressed":6355,"data_compressed_allocated":24576,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":188403,"data_compressed":4418,"data_compressed_allocated":20480,"data_compressed_original":90112,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:23:47.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:23:47.846 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.847 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:47.847 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:47.847 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:47.847 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:47.855 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8102 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.959 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.526862196, 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014232204, 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8102 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:47.960 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431821+0000", 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431820+0000", 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.961 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431830+0000", 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.7559999999999994e-06 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431837+0000", 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4880000000000004e-06 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431842+0000", 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.962 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445884+0000", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014042257000000001 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.445969+0000", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5037000000000006e-05 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446053+0000", 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.4147999999999995e-05 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.963 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8103 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52683714599999, 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014396289, 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8103 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.964 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431846+0000", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431845+0000", 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431848+0000", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.514e-06 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431855+0000", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1239999999999999e-06 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431860+0000", 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0599999999999998e-06 2024-09-21T14:23:47.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446094+0000", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014233915999999999 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446169+0000", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4748999999999999e-05 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446242+0000", 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3356999999999996e-05 2024-09-21T14:23:47.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8104 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.526818994, 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014593315000000001, 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8104 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.968 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431864+0000", 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431866+0000", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0889999999999998e-06 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431872+0000", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8109999999999999e-06 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431883+0000", 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1189000000000001e-05 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446284+0000", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014401295 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446373+0000", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.9414999999999997e-05 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446457+0000", 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3881000000000005e-05 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.971 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8105 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.526795757, 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747562000000001, 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8105 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431887+0000", 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431893+0000", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.173e-06 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431899+0000", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5289999999999997e-06 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431904+0000", 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6659999999999999e-06 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.974 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446499+0000", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014594630000000001 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446563+0000", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3818000000000005e-05 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446635+0000", 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2175999999999995e-05 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8106 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52677469, 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014879907, 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8106 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431908+0000", 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.977 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431910+0000", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9819999999999998e-06 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431915+0000", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.5079999999999996e-06 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431921+0000", 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3539999999999999e-06 2024-09-21T14:23:47.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446669+0000", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014747811 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446719+0000", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0878000000000003e-05 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446788+0000", 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8743000000000002e-05 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.979 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8107 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52675833000001, 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015082501, 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8107 2024-09-21T14:23:47.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431925+0000", 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.981 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431924+0000", 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431926+0000", 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0609999999999998e-06 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431933+0000", 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.982 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431937+0000", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2470000000000002e-06 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446843+0000", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014906215 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.446930+0000", 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.7058000000000006e-05 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447007+0000", 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7075000000000003e-05 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8108 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52672805, 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015192571, 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.984 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8108 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:47.985 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431955+0000", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431961+0000", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9109999999999998e-06 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431964+0000", 2024-09-21T14:23:47.986 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8639999999999998e-06 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431967+0000", 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6979999999999998e-06 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447039+0000", 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015071817 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.987 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447089+0000", 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9767e-05 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447148+0000", 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.8854000000000003e-05 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8109 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:47.988 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52671291199999, 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015333238000000001, 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8109 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431970+0000", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431976+0000", 2024-09-21T14:23:47.990 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3400000000000003e-06 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431983+0000", 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431987+0000", 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3409999999999996e-06 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.991 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447184+0000", 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015196724 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447256+0000", 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2532000000000003e-05 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447303+0000", 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7138000000000002e-05 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8110 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52669331600001, 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015426927, 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8110 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:47.993 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431990+0000", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431989+0000", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431991+0000", 2024-09-21T14:23:47.994 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.3969999999999999e-06 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431994+0000", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.1709999999999998e-06 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431996+0000", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.52e-06 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447335+0000", 2024-09-21T14:23:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015338203999999999 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447377+0000", 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2642999999999998e-05 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447417+0000", 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9314999999999999e-05 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8111 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.526684539, 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015585072, 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8111 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:47.997 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431998+0000", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.431999+0000", 2024-09-21T14:23:47.998 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.1289999999999999e-06 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432002+0000", 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8370000000000001e-06 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432008+0000", 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:47.999 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447448+0000", 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015440132000000001 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447508+0000", 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9759999999999997e-05 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447584+0000", 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.5362000000000001e-05 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8112 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52667217800001, 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015680229, 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8112 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.001 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432011+0000", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.002 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432012+0000", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.2020000000000001e-06 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432015+0000", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.977e-06 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432017+0000", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.3e-06 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.003 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447612+0000", 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015594502 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447654+0000", 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2707000000000003e-05 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447691+0000", 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6804000000000002e-05 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8113 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52666276100001, 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015774825999999999, 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8113 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432020+0000", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.006 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432022+0000", 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.7239999999999999e-06 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432028+0000", 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3849999999999999e-06 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432037+0000", 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.1309999999999995e-06 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.007 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447715+0000", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015678293999999999 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447755+0000", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.0076000000000001e-05 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447795+0000", 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9564999999999998e-05 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8114 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52664157000001, 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015862397, 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8114 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.009 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432041+0000", 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.010 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432043+0000", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0700000000000001e-06 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432049+0000", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432054+0000", 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7559999999999999e-06 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447819+0000", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015765159000000001 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447866+0000", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6823000000000002e-05 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447904+0000", 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7972000000000002e-05 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8115 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.526624462, 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015939266000000001, 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8115 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.013 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432059+0000", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432058+0000", 2024-09-21T14:23:48.014 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432060+0000", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0190000000000001e-06 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432067+0000", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.2999999999999998e-06 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432071+0000", 2024-09-21T14:23:48.015 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4109999999999998e-06 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447927+0000", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015855957 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447963+0000", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6489000000000002e-05 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.447998+0000", 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4430000000000001e-05 2024-09-21T14:23:48.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8116 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52660751900001, 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016012386, 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8116 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.017 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432075+0000", 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432077+0000", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9599999999999999e-06 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432083+0000", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.0460000000000002e-06 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432088+0000", 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5029999999999999e-06 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448018+0000", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015930782000000001 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448053+0000", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4458000000000001e-05 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448088+0000", 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4975999999999999e-05 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8117 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52659113999999, 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016085001000000002, 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8117 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432092+0000", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432091+0000", 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432094+0000", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0999999999999998e-06 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432108+0000", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.4272e-05 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432113+0000", 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.519e-06 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448108+0000", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.015994692000000001 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448141+0000", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3386000000000002e-05 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448177+0000", 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5404999999999997e-05 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8118 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52654978000001, 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016135215000000001, 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8118 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432133+0000", 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432135+0000", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.559e-06 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432143+0000", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3579999999999997e-06 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432148+0000", 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4870000000000002e-06 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448199+0000", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016050782 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448234+0000", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5481999999999997e-05 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448268+0000", 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.4115000000000001e-05 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:8119 3.4 3:210650b1:::rbd_data.11214feba615.000000000000005e:8 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.52653009900001, 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016201053999999999, 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 8119 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.029 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432153+0000", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432152+0000", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432155+0000", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1440000000000001e-06 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432160+0000", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.9699999999999996e-06 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.432165+0000", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4059999999999998e-06 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448282+0000", 2024-09-21T14:23:48.031 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016116973 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448321+0000", 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8683999999999998e-05 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.448354+0000", 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3405999999999998e-05 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46446 3.5 3:a6bc5b8c:::rbd_data.11214ba95254.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 115.12164165199999, 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.016811288000000001, 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46446 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:48.033 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837041+0000", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837040+0000", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.999999 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837047+0000", 2024-09-21T14:23:48.034 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.4989999999999999e-06 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837058+0000", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0709e-05 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837064+0000", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.533e-06 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837106+0000", 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1658e-05 2024-09-21T14:23:48.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.837183+0000", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.7191e-05 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 4", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.839984+0000", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0028006049999999998 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853773+0000", 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013789213999999999 2024-09-21T14:23:48.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853783+0000", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0379e-05 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853819+0000", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.5565999999999998e-05 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.853853+0000", 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.3865999999999997e-05 2024-09-21T14:23:48.037 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:93865 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 55.758870821000002, 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.953065563, 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 93865 2024-09-21T14:23:48.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199812+0000", 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.039 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199810+0000", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999981 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199817+0000", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7649999999999996e-06 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199837+0000", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9760000000000001e-05 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199846+0000", 2024-09-21T14:23:48.040 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.5180000000000004e-06 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199898+0000", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.2209000000000001e-05 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for peered", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:52.199906+0000", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.6570000000000003e-06 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151010+0000", 2024-09-21T14:23:48.041 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.95110341799999998 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151142+0000", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.000132409 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 5", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.151311+0000", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00016844599999999999 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152342+0000", 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.001031478 2024-09-21T14:23:48.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152769+0000", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.00042706599999999998 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152816+0000", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7042999999999997e-05 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:53.152878+0000", 2024-09-21T14:23:48.043 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.1478999999999999e-05 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.044 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:48.046 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:23:48.046 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:23:48.047 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:23:48.048 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-21T14:23:47.403148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107055327,"num_objects":481,"num_object_clones":34,"num_object_copies":962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":481,"num_whiteouts":414,"num_read":148362,"num_read_kb":954268,"num_write":21870,"num_write_kb":506140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19330,"ondisk_log_size":19330,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":357044,"kb_used_data":14532,"kb_used_omap":10,"kb_used_meta":342389,"kb_avail":660245836,"statfs":{"total":676457349120,"available":676091736064,"internally_reserved":0,"allocated":14880768,"data_stored":10280779,"data_compressed":198990,"data_compressed_allocated":413696,"data_compressed_original":2826240,"omap_allocated":11150,"internal_metadata":350606450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.248135"},"pg_stats":[{"pgid":"3.1f","version":"155'393","reported_seq":3662,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.261573+0000","last_change":"2024-09-21T14:23:42.223561+0000","last_active":"2024-09-21T14:23:42.261573+0000","last_peered":"2024-09-21T14:23:42.261573+0000","last_clean":"2024-09-21T14:23:42.261573+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:23:42.261573+0000","last_undegraded":"2024-09-21T14:23:42.261573+0000","last_fullsized":"2024-09-21T14:23:42.261573+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090032000000000003,"stat_sum":{"num_bytes":167675,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3820,"num_read_kb":17765,"num_write":359,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1e","version":"154'342","reported_seq":2484,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.222795+0000","last_change":"2024-09-21T14:23:42.223174+0000","last_active":"2024-09-21T14:23:42.222795+0000","last_peered":"2024-09-21T14:23:42.222795+0000","last_clean":"2024-09-21T14:23:42.222795+0000","last_became_active":"2024-09-21T14:23:33.163881+0000","last_became_peered":"2024-09-21T14:23:33.163881+0000","last_unstale":"2024-09-21T14:23:42.222795+0000","last_undegraded":"2024-09-21T14:23:42.222795+0000","last_fullsized":"2024-09-21T14:23:42.222795+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:57:22.066984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053884199999999999,"stat_sum":{"num_bytes":3207094,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":321,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1d","version":"155'449","reported_seq":3513,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258425+0000","last_change":"2024-09-21T14:23:42.161506+0000","last_active":"2024-09-21T14:23:42.258425+0000","last_peered":"2024-09-21T14:23:42.258425+0000","last_clean":"2024-09-21T14:23:42.258425+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:23:42.258425+0000","last_undegraded":"2024-09-21T14:23:42.258425+0000","last_fullsized":"2024-09-21T14:23:42.258425+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015082170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1c","version":"155'513","reported_seq":6296,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.261642+0000","last_change":"2024-09-21T14:23:42.171608+0000","last_active":"2024-09-21T14:23:42.261642+0000","last_peered":"2024-09-21T14:23:42.261642+0000","last_clean":"2024-09-21T14:23:42.261642+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:23:42.261642+0000","last_undegraded":"2024-09-21T14:23:42.261642+0000","last_fullsized":"2024-09-21T14:23:42.261642+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064302000000000001,"stat_sum":{"num_bytes":3995317,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":6528,"num_read_kb":83143,"num_write":550,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1b","version":"154'565","reported_seq":4967,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155462+0000","last_change":"2024-09-21T14:23:42.155799+0000","last_active":"2024-09-21T14:23:42.155462+0000","last_peered":"2024-09-21T14:23:42.155462+0000","last_clean":"2024-09-21T14:23:42.155462+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:23:42.155462+0000","last_undegraded":"2024-09-21T14:23:42.155462+0000","last_fullsized":"2024-09-21T14:23:42.155462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075175700000000003,"stat_sum":{"num_bytes":65536,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1a","version":"154'556","reported_seq":5765,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.301478+0000","last_change":"2024-09-21T14:23:42.171902+0000","last_active":"2024-09-21T14:23:42.301478+0000","last_peered":"2024-09-21T14:23:42.301478+0000","last_clean":"2024-09-21T14:23:42.301478+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:23:42.301478+0000","last_undegraded":"2024-09-21T14:23:42.301478+0000","last_fullsized":"2024-09-21T14:23:42.301478+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051669800000000001,"stat_sum":{"num_bytes":485403,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":13,"num_read":7186,"num_read_kb":66194,"num_write":578,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.19","version":"153'818","reported_seq":6126,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155915+0000","last_change":"2024-09-21T14:23:42.155915+0000","last_active":"2024-09-21T14:23:42.155915+0000","last_peered":"2024-09-21T14:23:42.155915+0000","last_clean":"2024-09-21T14:23:42.155915+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:23:42.155915+0000","last_undegraded":"2024-09-21T14:23:42.155915+0000","last_fullsized":"2024-09-21T14:23:42.155915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081868900000000005,"stat_sum":{"num_bytes":2142032,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":7452,"num_read_kb":30274,"num_write":1071,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.18","version":"155'296","reported_seq":2112,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.257112+0000","last_change":"2024-09-21T14:23:42.223942+0000","last_active":"2024-09-21T14:23:42.257112+0000","last_peered":"2024-09-21T14:23:42.257112+0000","last_clean":"2024-09-21T14:23:42.257112+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:23:42.257112+0000","last_undegraded":"2024-09-21T14:23:42.257112+0000","last_fullsized":"2024-09-21T14:23:42.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242579999999999,"stat_sum":{"num_bytes":471028,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.17","version":"155'833","reported_seq":3692,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258845+0000","last_change":"2024-09-21T14:23:42.223750+0000","last_active":"2024-09-21T14:23:42.258845+0000","last_peered":"2024-09-21T14:23:42.258845+0000","last_clean":"2024-09-21T14:23:42.258845+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:23:42.258845+0000","last_undegraded":"2024-09-21T14:23:42.258845+0000","last_fullsized":"2024-09-21T14:23:42.258845+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000884653,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.16","version":"155'529","reported_seq":2931,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.161759+0000","last_change":"2024-09-21T14:23:42.161759+0000","last_active":"2024-09-21T14:23:42.161759+0000","last_peered":"2024-09-21T14:23:42.161759+0000","last_clean":"2024-09-21T14:23:42.161759+0000","last_became_active":"2024-09-21T14:23:33.164931+0000","last_became_peered":"2024-09-21T14:23:33.164931+0000","last_unstale":"2024-09-21T14:23:42.161759+0000","last_undegraded":"2024-09-21T14:23:42.161759+0000","last_fullsized":"2024-09-21T14:23:42.161759+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:12.329268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001572101,"stat_sum":{"num_bytes":229376,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.15","version":"155'825","reported_seq":3063,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.203157+0000","last_change":"2024-09-21T14:23:42.203157+0000","last_active":"2024-09-21T14:23:42.203157+0000","last_peered":"2024-09-21T14:23:42.203157+0000","last_clean":"2024-09-21T14:23:42.203157+0000","last_became_active":"2024-09-21T14:23:24.295446+0000","last_became_peered":"2024-09-21T14:23:24.295446+0000","last_unstale":"2024-09-21T14:23:42.203157+0000","last_undegraded":"2024-09-21T14:23:42.203157+0000","last_fullsized":"2024-09-21T14:23:42.203157+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:09:36.195502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011680130000000001,"stat_sum":{"num_bytes":3336042,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.14","version":"155'514","reported_seq":3612,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.256136+0000","last_change":"2024-09-21T14:23:42.160467+0000","last_active":"2024-09-21T14:23:42.256136+0000","last_peered":"2024-09-21T14:23:42.256136+0000","last_clean":"2024-09-21T14:23:42.256136+0000","last_became_active":"2024-09-21T14:23:33.163697+0000","last_became_peered":"2024-09-21T14:23:33.163697+0000","last_unstale":"2024-09-21T14:23:42.256136+0000","last_undegraded":"2024-09-21T14:23:42.256136+0000","last_fullsized":"2024-09-21T14:23:42.256136+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:37:13.347634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058776599999999998,"stat_sum":{"num_bytes":163840,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154753+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:23:41.154753+0000","last_peered":"2024-09-21T14:23:41.154753+0000","last_clean":"2024-09-21T14:23:41.154753+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:23:41.154753+0000","last_undegraded":"2024-09-21T14:23:41.154753+0000","last_fullsized":"2024-09-21T14:23:41.154753+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'451","reported_seq":3170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.287665+0000","last_change":"2024-09-21T14:23:42.203455+0000","last_active":"2024-09-21T14:23:42.287665+0000","last_peered":"2024-09-21T14:23:42.287665+0000","last_clean":"2024-09-21T14:23:42.287665+0000","last_became_active":"2024-09-21T14:21:52.138165+0000","last_became_peered":"2024-09-21T14:21:52.138165+0000","last_unstale":"2024-09-21T14:23:42.287665+0000","last_undegraded":"2024-09-21T14:23:42.287665+0000","last_fullsized":"2024-09-21T14:23:42.287665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:04.411350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014078859999999999,"stat_sum":{"num_bytes":65552,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":17,"num_read":3583,"num_read_kb":12922,"num_write":409,"num_write_kb":5718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.172190+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:23:41.172190+0000","last_peered":"2024-09-21T14:23:41.172190+0000","last_clean":"2024-09-21T14:23:41.172190+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:23:41.172190+0000","last_undegraded":"2024-09-21T14:23:41.172190+0000","last_fullsized":"2024-09-21T14:23:41.172190+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"155'550","reported_seq":3398,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.202318+0000","last_change":"2024-09-21T14:23:42.202318+0000","last_active":"2024-09-21T14:23:42.202318+0000","last_peered":"2024-09-21T14:23:42.202318+0000","last_clean":"2024-09-21T14:23:42.202318+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:23:42.202318+0000","last_undegraded":"2024-09-21T14:23:42.202318+0000","last_fullsized":"2024-09-21T14:23:42.202318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049983999999999998,"stat_sum":{"num_bytes":2690959,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.157604+0000","last_change":"2024-09-21T14:23:30.327528+0000","last_active":"2024-09-21T14:23:41.157604+0000","last_peered":"2024-09-21T14:23:41.157604+0000","last_clean":"2024-09-21T14:23:41.157604+0000","last_became_active":"2024-09-21T14:23:30.327290+0000","last_became_peered":"2024-09-21T14:23:30.327290+0000","last_unstale":"2024-09-21T14:23:41.157604+0000","last_undegraded":"2024-09-21T14:23:41.157604+0000","last_fullsized":"2024-09-21T14:23:41.157604+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:32.836140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3278,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.155404+0000","last_change":"2024-09-21T14:23:42.155404+0000","last_active":"2024-09-21T14:23:42.155404+0000","last_peered":"2024-09-21T14:23:42.155404+0000","last_clean":"2024-09-21T14:23:42.155404+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:23:42.155404+0000","last_undegraded":"2024-09-21T14:23:42.155404+0000","last_fullsized":"2024-09-21T14:23:42.155404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042213500000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.4","version":"0'0","reported_seq":155,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.159947+0000","last_change":"2024-09-21T14:23:24.295710+0000","last_active":"2024-09-21T14:23:41.159947+0000","last_peered":"2024-09-21T14:23:41.159947+0000","last_clean":"2024-09-21T14:23:41.159947+0000","last_became_active":"2024-09-21T14:23:24.295425+0000","last_became_peered":"2024-09-21T14:23:24.295425+0000","last_unstale":"2024-09-21T14:23:41.159947+0000","last_undegraded":"2024-09-21T14:23:41.159947+0000","last_fullsized":"2024-09-21T14:23:41.159947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:55:26.597898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"155'582","reported_seq":5723,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.156110+0000","last_change":"2024-09-21T14:23:42.156110+0000","last_active":"2024-09-21T14:23:42.156110+0000","last_peered":"2024-09-21T14:23:42.156110+0000","last_clean":"2024-09-21T14:23:42.156110+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:23:42.156110+0000","last_undegraded":"2024-09-21T14:23:42.156110+0000","last_fullsized":"2024-09-21T14:23:42.156110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010170120000000001,"stat_sum":{"num_bytes":2572330,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":13,"num_read":6834,"num_read_kb":31516,"num_write":528,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.1","version":"155'385","reported_seq":2991,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.249192+0000","last_change":"2024-09-21T14:23:42.202326+0000","last_active":"2024-09-21T14:23:42.249192+0000","last_peered":"2024-09-21T14:23:42.249192+0000","last_clean":"2024-09-21T14:23:42.249192+0000","last_became_active":"2024-09-21T14:21:52.138768+0000","last_became_peered":"2024-09-21T14:21:52.138768+0000","last_unstale":"2024-09-21T14:23:42.249192+0000","last_undegraded":"2024-09-21T14:23:42.249192+0000","last_fullsized":"2024-09-21T14:23:42.249192+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:38.036055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051342900000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"2.0","version":"17'117","reported_seq":264,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.171245+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:23:41.171245+0000","last_peered":"2024-09-21T14:23:41.171245+0000","last_clean":"2024-09-21T14:23:41.171245+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:23:41.171245+0000","last_undegraded":"2024-09-21T14:23:41.171245+0000","last_fullsized":"2024-09-21T14:23:41.171245+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":180,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.161134+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:23:41.161134+0000","last_peered":"2024-09-21T14:23:41.161134+0000","last_clean":"2024-09-21T14:23:41.161134+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:23:41.161134+0000","last_undegraded":"2024-09-21T14:23:41.161134+0000","last_fullsized":"2024-09-21T14:23:41.161134+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"154'796","reported_seq":3478,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.223734+0000","last_change":"2024-09-21T14:23:42.224025+0000","last_active":"2024-09-21T14:23:42.223734+0000","last_peered":"2024-09-21T14:23:42.223734+0000","last_clean":"2024-09-21T14:23:42.223734+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:23:42.223734+0000","last_undegraded":"2024-09-21T14:23:42.223734+0000","last_fullsized":"2024-09-21T14:23:42.223734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010560610000000001,"stat_sum":{"num_bytes":16,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":5376,"num_read_kb":124714,"num_write":998,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.2","version":"17'2","reported_seq":42,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.161578+0000","last_change":"2024-09-21T14:23:24.298546+0000","last_active":"2024-09-21T14:23:41.161578+0000","last_peered":"2024-09-21T14:23:41.161578+0000","last_clean":"2024-09-21T14:23:41.161578+0000","last_became_active":"2024-09-21T14:23:24.298163+0000","last_became_peered":"2024-09-21T14:23:24.298163+0000","last_unstale":"2024-09-21T14:23:41.161578+0000","last_undegraded":"2024-09-21T14:23:41.161578+0000","last_fullsized":"2024-09-21T14:23:41.161578+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:52.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"155'1239","reported_seq":2928,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.284764+0000","last_change":"2024-09-21T14:23:42.202788+0000","last_active":"2024-09-21T14:23:42.284764+0000","last_peered":"2024-09-21T14:23:42.284764+0000","last_clean":"2024-09-21T14:23:42.284764+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:23:42.284764+0000","last_undegraded":"2024-09-21T14:23:42.284764+0000","last_fullsized":"2024-09-21T14:23:42.284764+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1239,"log_dups_size":0,"ondisk_log_size":1239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092164299999999998,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":4830,"num_read_kb":16531,"num_write":1688,"num_write_kb":8662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154780+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:23:41.154780+0000","last_peered":"2024-09-21T14:23:41.154780+0000","last_clean":"2024-09-21T14:23:41.154780+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:23:41.154780+0000","last_undegraded":"2024-09-21T14:23:41.154780+0000","last_fullsized":"2024-09-21T14:23:41.154780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"155'605","reported_seq":2780,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.253200+0000","last_change":"2024-09-21T14:23:42.156217+0000","last_active":"2024-09-21T14:23:42.253200+0000","last_peered":"2024-09-21T14:23:42.253200+0000","last_clean":"2024-09-21T14:23:42.253200+0000","last_became_active":"2024-09-21T14:21:52.138719+0000","last_became_peered":"2024-09-21T14:21:52.138719+0000","last_unstale":"2024-09-21T14:23:42.253200+0000","last_undegraded":"2024-09-21T14:23:42.253200+0000","last_fullsized":"2024-09-21T14:23:42.253200+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:26.264161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089169399999999995,"stat_sum":{"num_bytes":18190647,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":15,"num_read":3641,"num_read_kb":12437,"num_write":704,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T14:23:41.154608+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:23:41.154608+0000","last_peered":"2024-09-21T14:23:41.154608+0000","last_clean":"2024-09-21T14:23:41.154608+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:23:41.154608+0000","last_undegraded":"2024-09-21T14:23:41.154608+0000","last_fullsized":"2024-09-21T14:23:41.154608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"155'666","reported_seq":3718,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.307860+0000","last_change":"2024-09-21T14:23:42.155430+0000","last_active":"2024-09-21T14:23:42.307860+0000","last_peered":"2024-09-21T14:23:42.307860+0000","last_clean":"2024-09-21T14:23:42.307860+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:23:42.307860+0000","last_undegraded":"2024-09-21T14:23:42.307860+0000","last_fullsized":"2024-09-21T14:23:42.307860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056538900000000002,"stat_sum":{"num_bytes":180224,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":5855,"num_read_kb":16010,"num_write":714,"num_write_kb":12338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.9","version":"155'687","reported_seq":4744,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160053+0000","last_change":"2024-09-21T14:23:42.160053+0000","last_active":"2024-09-21T14:23:42.160053+0000","last_peered":"2024-09-21T14:23:42.160053+0000","last_clean":"2024-09-21T14:23:42.160053+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:23:42.160053+0000","last_undegraded":"2024-09-21T14:23:42.160053+0000","last_fullsized":"2024-09-21T14:23:42.160053+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038046199999999998,"stat_sum":{"num_bytes":38988713,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":12,"num_read":7645,"num_read_kb":84617,"num_write":703,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.a","version":"155'467","reported_seq":2298,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160427+0000","last_change":"2024-09-21T14:23:42.160427+0000","last_active":"2024-09-21T14:23:42.160427+0000","last_peered":"2024-09-21T14:23:42.160427+0000","last_clean":"2024-09-21T14:23:42.160427+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:23:42.160427+0000","last_undegraded":"2024-09-21T14:23:42.160427+0000","last_fullsized":"2024-09-21T14:23:42.160427+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057719100000000001,"stat_sum":{"num_bytes":36,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":1310,"num_read_kb":8590,"num_write":431,"num_write_kb":7229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.b","version":"155'474","reported_seq":2532,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.254086+0000","last_change":"2024-09-21T14:23:42.202766+0000","last_active":"2024-09-21T14:23:42.254086+0000","last_peered":"2024-09-21T14:23:42.254086+0000","last_clean":"2024-09-21T14:23:42.254086+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:23:42.254086+0000","last_undegraded":"2024-09-21T14:23:42.254086+0000","last_fullsized":"2024-09-21T14:23:42.254086+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089618900000000004,"stat_sum":{"num_bytes":11025616,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":2728,"num_read_kb":10420,"num_write":567,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.c","version":"155'740","reported_seq":2571,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.157539+0000","last_change":"2024-09-21T14:23:42.157539+0000","last_active":"2024-09-21T14:23:42.157539+0000","last_peered":"2024-09-21T14:23:42.157539+0000","last_clean":"2024-09-21T14:23:42.157539+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:23:42.157539+0000","last_undegraded":"2024-09-21T14:23:42.157539+0000","last_fullsized":"2024-09-21T14:23:42.157539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038724499999999998,"stat_sum":{"num_bytes":2313996,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3998,"num_read_kb":18124,"num_write":914,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.d","version":"155'566","reported_seq":3040,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.258670+0000","last_change":"2024-09-21T14:23:42.171832+0000","last_active":"2024-09-21T14:23:42.258670+0000","last_peered":"2024-09-21T14:23:42.258670+0000","last_clean":"2024-09-21T14:23:42.258670+0000","last_became_active":"2024-09-21T14:23:40.370380+0000","last_became_peered":"2024-09-21T14:23:40.370380+0000","last_unstale":"2024-09-21T14:23:42.258670+0000","last_undegraded":"2024-09-21T14:23:42.258670+0000","last_fullsized":"2024-09-21T14:23:42.258670+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:23.244070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066443000000000003,"stat_sum":{"num_bytes":4150664,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.e","version":"155'988","reported_seq":3867,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.305653+0000","last_change":"2024-09-21T14:23:42.223180+0000","last_active":"2024-09-21T14:23:42.305653+0000","last_peered":"2024-09-21T14:23:42.305653+0000","last_clean":"2024-09-21T14:23:42.305653+0000","last_became_active":"2024-09-21T14:23:24.298136+0000","last_became_peered":"2024-09-21T14:23:24.298136+0000","last_unstale":"2024-09-21T14:23:42.305653+0000","last_undegraded":"2024-09-21T14:23:42.305653+0000","last_fullsized":"2024-09-21T14:23:42.305653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:11:03.060692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055806699999999998,"stat_sum":{"num_bytes":167746,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":6087,"num_read_kb":20629,"num_write":1419,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.f","version":"155'489","reported_seq":5024,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.311014+0000","last_change":"2024-09-21T14:23:42.171585+0000","last_active":"2024-09-21T14:23:42.311014+0000","last_peered":"2024-09-21T14:23:42.311014+0000","last_clean":"2024-09-21T14:23:42.311014+0000","last_became_active":"2024-09-21T14:23:39.369052+0000","last_became_peered":"2024-09-21T14:23:39.369052+0000","last_unstale":"2024-09-21T14:23:42.311014+0000","last_undegraded":"2024-09-21T14:23:42.311014+0000","last_fullsized":"2024-09-21T14:23:42.311014+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:29.641991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041964900000000002,"stat_sum":{"num_bytes":1164045,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":16,"num_read":5799,"num_read_kb":25845,"num_write":455,"num_write_kb":21462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.10","version":"154'438","reported_seq":3543,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.171314+0000","last_change":"2024-09-21T14:23:42.171314+0000","last_active":"2024-09-21T14:23:42.171314+0000","last_peered":"2024-09-21T14:23:42.171314+0000","last_clean":"2024-09-21T14:23:42.171314+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:23:42.171314+0000","last_undegraded":"2024-09-21T14:23:42.171314+0000","last_fullsized":"2024-09-21T14:23:42.171314+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040741200000000001,"stat_sum":{"num_bytes":2490368,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.11","version":"155'381","reported_seq":3053,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.160439+0000","last_change":"2024-09-21T14:23:42.160439+0000","last_active":"2024-09-21T14:23:42.160439+0000","last_peered":"2024-09-21T14:23:42.160439+0000","last_clean":"2024-09-21T14:23:42.160439+0000","last_became_active":"2024-09-21T14:23:40.369251+0000","last_became_peered":"2024-09-21T14:23:40.369251+0000","last_unstale":"2024-09-21T14:23:42.160439+0000","last_undegraded":"2024-09-21T14:23:42.160439+0000","last_fullsized":"2024-09-21T14:23:42.160439+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:10:23.110922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041244399999999998,"stat_sum":{"num_bytes":4056861,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":18,"num_read":2072,"num_read_kb":23208,"num_write":411,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.12","version":"155'549","reported_seq":3731,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.253300+0000","last_change":"2024-09-21T14:23:42.203183+0000","last_active":"2024-09-21T14:23:42.253300+0000","last_peered":"2024-09-21T14:23:42.253300+0000","last_clean":"2024-09-21T14:23:42.253300+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:23:42.253300+0000","last_undegraded":"2024-09-21T14:23:42.253300+0000","last_fullsized":"2024-09-21T14:23:42.253300+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012320720000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"3"}]},{"pgid":"3.13","version":"155'1115","reported_seq":4835,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T14:23:42.279864+0000","last_change":"2024-09-21T14:23:42.161499+0000","last_active":"2024-09-21T14:23:42.279864+0000","last_peered":"2024-09-21T14:23:42.279864+0000","last_clean":"2024-09-21T14:23:42.279864+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:23:42.279864+0000","last_undegraded":"2024-09-21T14:23:42.279864+0000","last_fullsized":"2024-09-21T14:23:42.279864+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601119,"stat_sum":{"num_bytes":4143783,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":14,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":106464940,"num_objects":477,"num_object_clones":34,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":414,"num_read":148246,"num_read_kb":954168,"num_write":21709,"num_write_kb":503722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5279,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3334144,"data_stored":17706081,"data_compressed":922618,"data_compressed_allocated":1351680,"data_compressed_original":16193563,"omap_allocated":0,"internal_metadata":0},"log_size":19210,"ondisk_log_size":19210,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2068480,"data_stored":1353789,"data_compressed":31279,"data_compressed_allocated":61440,"data_compressed_original":327680,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.437},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.496},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.375}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2142208,"data_stored":1396311,"data_compressed":22731,"data_compressed_allocated":61440,"data_compressed_original":311296,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.435}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.46800000000000003}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35499999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.37},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55364,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316476,"statfs":{"total":96636764160,"available":96580071424,"internally_reserved":0,"allocated":2150400,"data_stored":1730576,"data_compressed":35148,"data_compressed_allocated":57344,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.32100000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.372},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.36399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.501}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.438}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":2129920,"data_stored":1350001,"data_compressed":25375,"data_compressed_allocated":61440,"data_compressed_original":286720,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.624},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68500000000000005}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69799999999999995}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59484,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312356,"statfs":{"total":96636764160,"available":96575852544,"internally_reserved":0,"allocated":2170880,"data_stored":1898159,"data_compressed":49158,"data_compressed_allocated":86016,"data_compressed_original":823296,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.315},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.502},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72499999999999998}]}]},{"osd":2,"up_from":0,"seq":55834574881,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":2064384,"data_stored":1122455,"data_compressed":4016,"data_compressed_allocated":16384,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.252}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.221},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.31}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.28699999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41299999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.437},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.52400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65224,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":2154496,"data_stored":1429488,"data_compressed":31283,"data_compressed_allocated":69632,"data_compressed_original":364544,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.312}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":237105,"data_compressed":5298,"data_compressed_allocated":24576,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":214058,"data_compressed":4016,"data_compressed_allocated":16384,"data_compressed_original":81920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":16074715,"data_compressed":888166,"data_compressed_allocated":1212416,"data_compressed_original":15480859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":281190,"data_compressed":7125,"data_compressed_allocated":24576,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":229970,"data_compressed":4902,"data_compressed_allocated":20480,"data_compressed_original":102400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":204892,"data_compressed":2338,"data_compressed_allocated":8192,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":275748,"data_compressed":6355,"data_compressed_allocated":24576,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":188403,"data_compressed":4418,"data_compressed_allocated":20480,"data_compressed_original":90112,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:23:48.051 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:23:48.051 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:23:48.051 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-21T14:23:48.051 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-21T14:23:48.154 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:48.154 INFO:teuthology.orchestra.run.smithi161.stdout: "num_ops": 0 2024-09-21T14:23:48.154 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:48.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:23:48.264 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [], 2024-09-21T14:23:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "complaint_time": 30, 2024-09-21T14:23:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "num_blocked_ops": 0 2024-09-21T14:23:48.264 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:48.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:23:48.332 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-21T14:23:48.332+0000 7f0ce787f640 -1 mon.a@0(leader).osd e160 definitely_dead 0 2024-09-21T14:23:48.376 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T14:23:48.376 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 20, 2024-09-21T14:23:48.376 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 600, 2024-09-21T14:23:48.376 INFO:teuthology.orchestra.run.smithi161.stdout: "ops": [ 2024-09-21T14:23:48.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9534 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77973248800001, 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018442204, 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9534 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.377 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596414+0000", 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596413+0000", 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596415+0000", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1459999999999999e-06 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596422+0000", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1189999999999999e-06 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596427+0000", 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4839999999999998e-06 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.379 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614756+0000", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018329227999999999 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614805+0000", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8687000000000003e-05 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614856+0000", 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0905999999999997e-05 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9535 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.779714974, 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.01863275, 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9535 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.381 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596431+0000", 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596433+0000", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9649999999999998e-06 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596439+0000", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7410000000000003e-06 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596444+0000", 2024-09-21T14:23:48.383 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4240000000000003e-06 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614899+0000", 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018454990000000001 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.614983+0000", 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3886000000000003e-05 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615064+0000", 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1083999999999998e-05 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9536 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77969772, 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018820552000000001, 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.385 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9536 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:48.386 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596448+0000", 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596453+0000", 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.3990000000000003e-06 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.387 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596461+0000", 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.542e-06 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596472+0000", 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0789e-05 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615109+0000", 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018636974000000001 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615189+0000", 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0445000000000007e-05 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615269+0000", 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9771999999999994e-05 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.389 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9537 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77966986999999, 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018916408999999999, 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9537 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.390 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596476+0000", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.391 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596481+0000", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7049999999999996e-06 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596489+0000", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2110000000000001e-06 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596493+0000", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.245e-06 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615301+0000", 2024-09-21T14:23:48.392 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018808065999999998 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615345+0000", 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3807999999999998e-05 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615393+0000", 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7854e-05 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9538 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77964826600001, 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019088899999999999, 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9538 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596498+0000", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596497+0000", 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596500+0000", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1339999999999998e-06 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596506+0000", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.861e-06 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596511+0000", 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3649999999999997e-06 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615431+0000", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018920717 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615512+0000", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.0442e-05 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615587+0000", 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4785999999999999e-05 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.397 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9539 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 477184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.779631268, 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019275055999999999, 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9539 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.398 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596515+0000", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596514+0000", 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596516+0000", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.8959999999999999e-06 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596523+0000", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7159999999999997e-06 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596527+0000", 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3069999999999996e-06 2024-09-21T14:23:48.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615630+0000", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019102622 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615711+0000", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1434999999999995e-05 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615790+0000", 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.8458e-05 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.401 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9540 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.779614248, 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019447684, 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9540 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.402 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596532+0000", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596531+0000", 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596534+0000", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.052e-06 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596540+0000", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596545+0000", 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.4800000000000003e-06 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615834+0000", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019289494000000001 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615908+0000", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.4091000000000006e-05 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.615980+0000", 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1037000000000004e-05 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9541 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.779596798, 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019555014999999999, 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9541 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.406 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596549+0000", 2024-09-21T14:23:48.407 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596551+0000", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.001e-06 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596558+0000", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6590000000000001e-06 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596562+0000", 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5430000000000004e-06 2024-09-21T14:23:48.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616011+0000", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019448664000000001 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616053+0000", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2577000000000002e-05 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616104+0000", 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.0912999999999999e-05 2024-09-21T14:23:48.409 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9542 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77957966, 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019659141000000002, 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.410 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9542 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596566+0000", 2024-09-21T14:23:48.411 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596568+0000", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1409999999999999e-06 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596575+0000", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8569999999999997e-06 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596585+0000", 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0173000000000001e-05 2024-09-21T14:23:48.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616136+0000", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019550732000000001 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616183+0000", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6981000000000003e-05 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616226+0000", 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2667000000000003e-05 2024-09-21T14:23:48.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9543 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77955643300001, 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019733753999999999, 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9543 2024-09-21T14:23:48.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596590+0000", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596589+0000", 2024-09-21T14:23:48.415 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596591+0000", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1430000000000001e-06 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596597+0000", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.677e-06 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596602+0000", 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5210000000000004e-06 2024-09-21T14:23:48.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616236+0000", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019634671999999999 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616285+0000", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8535000000000002e-05 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616323+0000", 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.417 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9544 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77954015899999, 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019891643000000001, 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9544 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.418 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596606+0000", 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596614+0000", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.1380000000000007e-06 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596621+0000", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.2060000000000001e-06 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596625+0000", 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6210000000000003e-06 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.420 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616358+0000", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019732544000000001 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616431+0000", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.3112000000000001e-05 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616498+0000", 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6419000000000002e-05 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9545 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 501760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77951578700001, 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020004948000000002, 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9545 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.422 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596630+0000", 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.423 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596632+0000", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.0940000000000002e-06 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596639+0000", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.7789999999999998e-06 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596648+0000", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5789999999999993e-06 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.424 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616535+0000", 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019886576 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616583+0000", 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8420000000000001e-05 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616635+0000", 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.1894000000000002e-05 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9546 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77949322000001, 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020084712000000001, 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9546 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596653+0000", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596652+0000", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596655+0000", 2024-09-21T14:23:48.427 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.114e-06 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596661+0000", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6529999999999997e-06 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596666+0000", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.2740000000000001e-06 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616661+0000", 2024-09-21T14:23:48.428 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.019995242999999999 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616700+0000", 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8733999999999999e-05 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616738+0000", 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.8077999999999997e-05 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.429 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9547 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77947629900001, 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020187428, 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9547 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:48.430 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596670+0000", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596669+0000", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596671+0000", 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.9910000000000001e-06 2024-09-21T14:23:48.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596678+0000", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.6619999999999999e-06 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596682+0000", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1289999999999998e-06 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616760+0000", 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020078054000000001 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616808+0000", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7343999999999999e-05 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616857+0000", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.9620000000000003e-05 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9548 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:48.433 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77945980000001, 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020342373, 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9548 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596686+0000", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596691+0000", 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.4269999999999999e-06 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.435 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596698+0000", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.9e-06 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596703+0000", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.6269999999999999e-06 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616898+0000", 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020195379999999999 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.436 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.616982+0000", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 8.3249000000000002e-05 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617029+0000", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.7151999999999998e-05 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9549 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77943880300001, 2024-09-21T14:23:48.437 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020487783999999998, 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9549 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.438 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596707+0000", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596713+0000", 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.3090000000000001e-06 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.439 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596720+0000", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.1509999999999998e-06 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596725+0000", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.5750000000000002e-06 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617060+0000", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.02033548 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617129+0000", 2024-09-21T14:23:48.440 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.8873000000000003e-05 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617195+0000", 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 6.5826000000000003e-05 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:9550 3.b 3:d014cf2f:::rbd_data.11214feba615.000000000000004c:8 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 138.77941674499999, 2024-09-21T14:23:48.441 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020570773000000001, 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "started", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 9550 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.442 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596729+0000", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596731+0000", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.148e-06 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.443 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596738+0000", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.109e-06 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.596743+0000", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.3889999999999998e-06 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617217+0000", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.020474307000000001 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.444 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617258+0000", 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:29.617300+0000", 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.1838999999999997e-05 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_op(client.4385.0:46387 3.b 3:d0d9ddc8:::rbd_data.11214ba95254.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 115.605345873, 2024-09-21T14:23:48.445 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.022625711, 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "client_info": { 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "client": "client.4385", 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "client_addr": "172.21.15.73:0/1834386442", 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "tid": 46387 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770800+0000", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770798+0000", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999976 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770808+0000", 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0387000000000001e-05 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.447 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770837+0000", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.9101000000000001e-05 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770848+0000", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.0231000000000001e-05 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770885+0000", 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7697000000000002e-05 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.448 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.770942+0000", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.6347000000000001e-05 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "waiting for subops from 6", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.775707+0000", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.0047655730000000004 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "op_commit", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793322+0000", 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.017614736999999998 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793360+0000", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7929000000000002e-05 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793398+0000", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.7883000000000003e-05 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:21:52.793426+0000", 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.8127e-05 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.450 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:80693 3.0 e77/19 3:04f72909:::rbd_data.1121590fed9b.0000000000000003:head v 77'598, mlcod=72'596)", 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 84.463112749000004, 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.03323628, 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.451 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.913033+0000", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967296 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928009+0000", 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.014975989 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.452 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928031+0000", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.1951999999999999e-05 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928041+0000", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 9.5289999999999997e-06 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928090+0000", 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4.8953999999999999e-05 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.453 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.928116+0000", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6526e-05 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946233+0000", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.018117286999999999 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:23.946270+0000", 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.6202000000000003e-05 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.454 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "description": "osd_repop(client.4385.0:82787 3.0 e78/19 3:04c56297:::rbd_data.1121a2b38c70.0000000000000003:head v 78'601, mlcod=77'599)", 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "initiated_at": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "age": 83.230402673, 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.025760468000000002, 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "type_data": { 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "events": [ 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "initiated", 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.455 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "throttled", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "header_read", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.145743+0000", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 4294967295.9999995 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "all_read", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158845+0000", 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.013101859 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.456 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "dispatched", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158861+0000", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 1.5950000000000001e-05 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "queued_for_pg", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158869+0000", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 7.9000000000000006e-06 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "reached_pg", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158926+0000", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 5.7673999999999997e-05 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "started", 2024-09-21T14:23:48.457 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.158953+0000", 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 2.6424e-05 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "commit_sent", 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171464+0000", 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 0.012511426000000001 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "event": "done", 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "time": "2024-09-21T14:22:25.171504+0000", 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: "duration": 3.9586999999999999e-05 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T14:23:48.459 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T14:23:48.459 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T14:23:48.459 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.2. 2024-09-21T14:23:48.460 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-21T14:23:48.461 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-21T14:23:53.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T14:23:53.587 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-09-21T14:23:53.587 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:23:53.597 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-09-21T14:23:53.597 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:23:53.605 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-09-21T14:23:53.605 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:23:53.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:23:54.162 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:23:54.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T14:23:55.168 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T14:23:55.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T14:23:56.170 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T14:23:56.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T14:23:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T14:23:57.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T14:23:58.187 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T14:23:58.204 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T14:23:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T14:23:59.209 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T14:24:00.199 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T14:24:00.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T14:24:01.204 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T14:24:01.221 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:24:02.211 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:24:02.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T14:24:03.220 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T14:24:03.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:24:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:24:04.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T14:24:05.228 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T14:24:05.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:24:06.238 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:24:06.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T14:24:07.245 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T14:24:07.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:24:08.259 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:24:08.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-21T14:24:09.266 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T14:24:09.282 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T14:24:09.283 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T14:24:09.283 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T14:24:09.283 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T14:24:09.283 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T14:24:09.283 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:24:09.557 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:09.557 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":182,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:24:09.263377+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:23:48.408443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-21T14:21:07.255609+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-21T14:21:08.449794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:21:16.638770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":97,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.729613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.843490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.767547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.797830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.760985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.830324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.747245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.849223+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.1","osds":[5,4]},{"pgid":"3.3","osds":[6,5]},{"pgid":"3.d","osds":[6,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:24:09.570 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T14:24:09.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:24:09.841 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:09.841 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":182,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:24:09.263377+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:23:48.408443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-21T14:21:07.255609+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-21T14:21:08.449794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:21:16.638770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":97,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.729613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.843490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.767547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.797830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.760985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.830324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.747245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.849223+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.1","osds":[5,4]},{"pgid":"3.3","osds":[6,5]},{"pgid":"3.d","osds":[6,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:24:09.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T14:24:09.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T14:24:09.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T14:24:09.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T14:24:09.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T14:24:09.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T14:24:09.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T14:24:09.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T14:24:10.080 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:24:10.090 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:10.091 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:24:10.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:24:10.097 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:10.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:24:10.098 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:10.098 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:24:10.099 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:24:10.100 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:24:10.101 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:10.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:24:10.614 INFO:teuthology.orchestra.run.smithi073.stdout:55834574886 2024-09-21T14:24:10.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.2 2024-09-21T14:24:10.635 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.640 INFO:teuthology.orchestra.run.smithi073.stdout:55834574886 2024-09-21T14:24:10.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.3 2024-09-21T14:24:10.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.4 2024-09-21T14:24:10.671 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.682 INFO:teuthology.orchestra.run.smithi073.stdout:55834574886 2024-09-21T14:24:10.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.0 2024-09-21T14:24:10.688 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.6 2024-09-21T14:24:10.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.7 2024-09-21T14:24:10.724 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:10.728 INFO:teuthology.orchestra.run.smithi073.stdout:55834574886 2024-09-21T14:24:10.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.1 2024-09-21T14:24:10.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.5 2024-09-21T14:24:11.635 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:24:11.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:24:11.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:24:11.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:24:11.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:24:11.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:24:11.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:24:11.741 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:24:12.139 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:12.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.2 2024-09-21T14:24:12.159 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.207 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:12.220 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:12.223 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:12.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.4 2024-09-21T14:24:12.226 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.6 2024-09-21T14:24:12.235 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.3 2024-09-21T14:24:12.243 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.263 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:12.269 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:12.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.0 2024-09-21T14:24:12.277 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.1 2024-09-21T14:24:12.283 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.308 INFO:teuthology.orchestra.run.smithi073.stdout:55834574888 2024-09-21T14:24:12.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.7 2024-09-21T14:24:12.321 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.325 INFO:teuthology.orchestra.run.smithi073.stdout:55834574887 2024-09-21T14:24:12.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.5 2024-09-21T14:24:12.338 DEBUG:teuthology.parallel:result is None 2024-09-21T14:24:12.338 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:24:12.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:12.665 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T14:24:12.447395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399012,"kb_used_data":17200,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575708,"statfs":{"total":773094113280,"available":772685524992,"internally_reserved":0,"allocated":17612800,"data_stored":9720000,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200102"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3711,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626213+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:07.626213+0000","last_peered":"2024-09-21T14:24:07.626213+0000","last_clean":"2024-09-21T14:24:07.626213+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:07.626213+0000","last_undegraded":"2024-09-21T14:24:07.626213+0000","last_fullsized":"2024-09-21T14:24:07.626213+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2728,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626140+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:07.626140+0000","last_peered":"2024-09-21T14:24:07.626140+0000","last_clean":"2024-09-21T14:24:07.626140+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:07.626140+0000","last_undegraded":"2024-09-21T14:24:07.626140+0000","last_fullsized":"2024-09-21T14:24:07.626140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3558,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259240+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:08.259240+0000","last_peered":"2024-09-21T14:24:08.259240+0000","last_clean":"2024-09-21T14:24:08.259240+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:08.259240+0000","last_undegraded":"2024-09-21T14:24:08.259240+0000","last_fullsized":"2024-09-21T14:24:08.259240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6356,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572964+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:08.572964+0000","last_peered":"2024-09-21T14:24:08.572964+0000","last_clean":"2024-09-21T14:24:08.572964+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:08.572964+0000","last_undegraded":"2024-09-21T14:24:08.572964+0000","last_fullsized":"2024-09-21T14:24:08.572964+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573806+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:08.573806+0000","last_peered":"2024-09-21T14:24:08.573806+0000","last_clean":"2024-09-21T14:24:08.573806+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:08.573806+0000","last_undegraded":"2024-09-21T14:24:08.573806+0000","last_fullsized":"2024-09-21T14:24:08.573806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572962+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:08.572962+0000","last_peered":"2024-09-21T14:24:08.572962+0000","last_clean":"2024-09-21T14:24:08.572962+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:08.572962+0000","last_undegraded":"2024-09-21T14:24:08.572962+0000","last_fullsized":"2024-09-21T14:24:08.572962+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6266,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573965+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:08.573965+0000","last_peered":"2024-09-21T14:24:08.573965+0000","last_clean":"2024-09-21T14:24:08.573965+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:08.573965+0000","last_undegraded":"2024-09-21T14:24:08.573965+0000","last_fullsized":"2024-09-21T14:24:08.573965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625986+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:07.625986+0000","last_peered":"2024-09-21T14:24:07.625986+0000","last_clean":"2024-09-21T14:24:07.625986+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:07.625986+0000","last_undegraded":"2024-09-21T14:24:07.625986+0000","last_fullsized":"2024-09-21T14:24:07.625986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3736,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626282+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:07.626282+0000","last_peered":"2024-09-21T14:24:07.626282+0000","last_clean":"2024-09-21T14:24:07.626282+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:07.626282+0000","last_undegraded":"2024-09-21T14:24:07.626282+0000","last_fullsized":"2024-09-21T14:24:07.626282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2983,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248820+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:07.248820+0000","last_peered":"2024-09-21T14:24:07.248820+0000","last_clean":"2024-09-21T14:24:07.248820+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:07.248820+0000","last_undegraded":"2024-09-21T14:24:07.248820+0000","last_fullsized":"2024-09-21T14:24:07.248820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:14.583949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625177+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:07.625177+0000","last_peered":"2024-09-21T14:24:07.625177+0000","last_clean":"2024-09-21T14:24:07.625177+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:07.625177+0000","last_undegraded":"2024-09-21T14:24:07.625177+0000","last_fullsized":"2024-09-21T14:24:07.625177+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:15.056601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259283+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:08.259283+0000","last_peered":"2024-09-21T14:24:08.259283+0000","last_clean":"2024-09-21T14:24:08.259283+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:08.259283+0000","last_undegraded":"2024-09-21T14:24:08.259283+0000","last_fullsized":"2024-09-21T14:24:08.259283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:34.912969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573015+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:08.573015+0000","last_peered":"2024-09-21T14:24:08.573015+0000","last_clean":"2024-09-21T14:24:08.573015+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:08.573015+0000","last_undegraded":"2024-09-21T14:24:08.573015+0000","last_fullsized":"2024-09-21T14:24:08.573015+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3306,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248798+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:07.248798+0000","last_peered":"2024-09-21T14:24:07.248798+0000","last_clean":"2024-09-21T14:24:07.248798+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:07.248798+0000","last_undegraded":"2024-09-21T14:24:07.248798+0000","last_fullsized":"2024-09-21T14:24:07.248798+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.949047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259426+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:08.259426+0000","last_peered":"2024-09-21T14:24:08.259426+0000","last_clean":"2024-09-21T14:24:08.259426+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:08.259426+0000","last_undegraded":"2024-09-21T14:24:08.259426+0000","last_fullsized":"2024-09-21T14:24:08.259426+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3446,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626177+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:07.626177+0000","last_peered":"2024-09-21T14:24:07.626177+0000","last_clean":"2024-09-21T14:24:07.626177+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:07.626177+0000","last_undegraded":"2024-09-21T14:24:07.626177+0000","last_fullsized":"2024-09-21T14:24:07.626177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248822+0000","last_change":"2024-09-21T14:23:49.431117+0000","last_active":"2024-09-21T14:24:07.248822+0000","last_peered":"2024-09-21T14:24:07.248822+0000","last_clean":"2024-09-21T14:24:07.248822+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:07.248822+0000","last_undegraded":"2024-09-21T14:24:07.248822+0000","last_fullsized":"2024-09-21T14:24:07.248822+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:12.106526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3322,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574105+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:08.574105+0000","last_peered":"2024-09-21T14:24:08.574105+0000","last_clean":"2024-09-21T14:24:08.574105+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:08.574105+0000","last_undegraded":"2024-09-21T14:24:08.574105+0000","last_fullsized":"2024-09-21T14:24:08.574105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":150,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625138+0000","last_change":"2024-09-21T14:23:56.392702+0000","last_active":"2024-09-21T14:24:07.625138+0000","last_peered":"2024-09-21T14:24:07.625138+0000","last_clean":"2024-09-21T14:24:07.625138+0000","last_became_active":"2024-09-21T14:23:56.392416+0000","last_became_peered":"2024-09-21T14:23:56.392416+0000","last_unstale":"2024-09-21T14:24:07.625138+0000","last_undegraded":"2024-09-21T14:24:07.625138+0000","last_fullsized":"2024-09-21T14:24:07.625138+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:58.517054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5776,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574132+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:08.574132+0000","last_peered":"2024-09-21T14:24:08.574132+0000","last_clean":"2024-09-21T14:24:08.574132+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:08.574132+0000","last_undegraded":"2024-09-21T14:24:08.574132+0000","last_fullsized":"2024-09-21T14:24:08.574132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625872+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:07.625872+0000","last_peered":"2024-09-21T14:24:07.625872+0000","last_clean":"2024-09-21T14:24:07.625872+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:07.625872+0000","last_undegraded":"2024-09-21T14:24:07.625872+0000","last_fullsized":"2024-09-21T14:24:07.625872+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":291,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.258998+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:08.258998+0000","last_peered":"2024-09-21T14:24:08.258998+0000","last_clean":"2024-09-21T14:24:08.258998+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:08.258998+0000","last_undegraded":"2024-09-21T14:24:08.258998+0000","last_fullsized":"2024-09-21T14:24:08.258998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":206,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573907+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:08.573907+0000","last_peered":"2024-09-21T14:24:08.573907+0000","last_clean":"2024-09-21T14:24:08.573907+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:08.573907+0000","last_undegraded":"2024-09-21T14:24:08.573907+0000","last_fullsized":"2024-09-21T14:24:08.573907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3525,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625976+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:07.625976+0000","last_peered":"2024-09-21T14:24:07.625976+0000","last_clean":"2024-09-21T14:24:07.625976+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:07.625976+0000","last_undegraded":"2024-09-21T14:24:07.625976+0000","last_fullsized":"2024-09-21T14:24:07.625976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.624926+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:07.624926+0000","last_peered":"2024-09-21T14:24:07.624926+0000","last_clean":"2024-09-21T14:24:07.624926+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:07.624926+0000","last_undegraded":"2024-09-21T14:24:07.624926+0000","last_fullsized":"2024-09-21T14:24:07.624926+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:46.419594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3006,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625807+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:07.625807+0000","last_peered":"2024-09-21T14:24:07.625807+0000","last_clean":"2024-09-21T14:24:07.625807+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:07.625807+0000","last_undegraded":"2024-09-21T14:24:07.625807+0000","last_fullsized":"2024-09-21T14:24:07.625807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626070+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:07.626070+0000","last_peered":"2024-09-21T14:24:07.626070+0000","last_clean":"2024-09-21T14:24:07.626070+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:07.626070+0000","last_undegraded":"2024-09-21T14:24:07.626070+0000","last_fullsized":"2024-09-21T14:24:07.626070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3053,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573765+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:08.573765+0000","last_peered":"2024-09-21T14:24:08.573765+0000","last_clean":"2024-09-21T14:24:08.573765+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:08.573765+0000","last_undegraded":"2024-09-21T14:24:08.573765+0000","last_fullsized":"2024-09-21T14:24:08.573765+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572963+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:08.572963+0000","last_peered":"2024-09-21T14:24:08.572963+0000","last_clean":"2024-09-21T14:24:08.572963+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:08.572963+0000","last_undegraded":"2024-09-21T14:24:08.572963+0000","last_fullsized":"2024-09-21T14:24:08.572963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3831,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573600+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:08.573600+0000","last_peered":"2024-09-21T14:24:08.573600+0000","last_clean":"2024-09-21T14:24:08.573600+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:08.573600+0000","last_undegraded":"2024-09-21T14:24:08.573600+0000","last_fullsized":"2024-09-21T14:24:08.573600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4809,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259111+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:08.259111+0000","last_peered":"2024-09-21T14:24:08.259111+0000","last_clean":"2024-09-21T14:24:08.259111+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:08.259111+0000","last_undegraded":"2024-09-21T14:24:08.259111+0000","last_fullsized":"2024-09-21T14:24:08.259111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259152+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:08.259152+0000","last_peered":"2024-09-21T14:24:08.259152+0000","last_clean":"2024-09-21T14:24:08.259152+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:08.259152+0000","last_undegraded":"2024-09-21T14:24:08.259152+0000","last_fullsized":"2024-09-21T14:24:08.259152+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2582,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626037+0000","last_change":"2024-09-21T14:23:46.158494+0000","last_active":"2024-09-21T14:24:07.626037+0000","last_peered":"2024-09-21T14:24:07.626037+0000","last_clean":"2024-09-21T14:24:07.626037+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:24:07.626037+0000","last_undegraded":"2024-09-21T14:24:07.626037+0000","last_fullsized":"2024-09-21T14:24:07.626037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894741,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2618,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625057+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:07.625057+0000","last_peered":"2024-09-21T14:24:07.625057+0000","last_clean":"2024-09-21T14:24:07.625057+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:07.625057+0000","last_undegraded":"2024-09-21T14:24:07.625057+0000","last_fullsized":"2024-09-21T14:24:07.625057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626196+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:07.626196+0000","last_peered":"2024-09-21T14:24:07.626196+0000","last_clean":"2024-09-21T14:24:07.626196+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:07.626196+0000","last_undegraded":"2024-09-21T14:24:07.626196+0000","last_fullsized":"2024-09-21T14:24:07.626196+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3911,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248806+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:07.248806+0000","last_peered":"2024-09-21T14:24:07.248806+0000","last_clean":"2024-09-21T14:24:07.248806+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:07.248806+0000","last_undegraded":"2024-09-21T14:24:07.248806+0000","last_fullsized":"2024-09-21T14:24:07.248806+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:32.841741+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625092+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:07.625092+0000","last_peered":"2024-09-21T14:24:07.625092+0000","last_clean":"2024-09-21T14:24:07.625092+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:07.625092+0000","last_undegraded":"2024-09-21T14:24:07.625092+0000","last_fullsized":"2024-09-21T14:24:07.625092+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:05.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3588,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572927+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:08.572927+0000","last_peered":"2024-09-21T14:24:08.572927+0000","last_clean":"2024-09-21T14:24:08.572927+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:08.572927+0000","last_undegraded":"2024-09-21T14:24:08.572927+0000","last_fullsized":"2024-09-21T14:24:08.572927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259197+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:08.259197+0000","last_peered":"2024-09-21T14:24:08.259197+0000","last_clean":"2024-09-21T14:24:08.259197+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:08.259197+0000","last_undegraded":"2024-09-21T14:24:08.259197+0000","last_fullsized":"2024-09-21T14:24:08.259197+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:21.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3775,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625902+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:07.625902+0000","last_peered":"2024-09-21T14:24:07.625902+0000","last_clean":"2024-09-21T14:24:07.625902+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:07.625902+0000","last_undegraded":"2024-09-21T14:24:07.625902+0000","last_fullsized":"2024-09-21T14:24:07.625902+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259038+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:08.259038+0000","last_peered":"2024-09-21T14:24:08.259038+0000","last_clean":"2024-09-21T14:24:08.259038+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:08.259038+0000","last_undegraded":"2024-09-21T14:24:08.259038+0000","last_fullsized":"2024-09-21T14:24:08.259038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.497}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2248704,"data_stored":1655571,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59548,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312292,"statfs":{"total":96636764160,"available":96575787008,"internally_reserved":0,"allocated":2244608,"data_stored":1662438,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.499},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.442}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29160,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342680,"statfs":{"total":96636764160,"available":96606904320,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.34399999999999997,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.44800000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.38500000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46899999999999997,"5min":0.621,"15min":0.621},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.44900000000000001,"15min":0.46899999999999997},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.55800000000000005,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.442},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":2.097,"15min":2.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":1071103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.251},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.308}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65252,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306588,"statfs":{"total":96636764160,"available":96569946112,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:12.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:12.940 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:12.940 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:12.951 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T14:24:12.447395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399012,"kb_used_data":17200,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575708,"statfs":{"total":773094113280,"available":772685524992,"internally_reserved":0,"allocated":17612800,"data_stored":9720000,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200102"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3711,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626213+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:07.626213+0000","last_peered":"2024-09-21T14:24:07.626213+0000","last_clean":"2024-09-21T14:24:07.626213+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:07.626213+0000","last_undegraded":"2024-09-21T14:24:07.626213+0000","last_fullsized":"2024-09-21T14:24:07.626213+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2728,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626140+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:07.626140+0000","last_peered":"2024-09-21T14:24:07.626140+0000","last_clean":"2024-09-21T14:24:07.626140+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:07.626140+0000","last_undegraded":"2024-09-21T14:24:07.626140+0000","last_fullsized":"2024-09-21T14:24:07.626140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3558,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259240+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:08.259240+0000","last_peered":"2024-09-21T14:24:08.259240+0000","last_clean":"2024-09-21T14:24:08.259240+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:08.259240+0000","last_undegraded":"2024-09-21T14:24:08.259240+0000","last_fullsized":"2024-09-21T14:24:08.259240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6356,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572964+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:08.572964+0000","last_peered":"2024-09-21T14:24:08.572964+0000","last_clean":"2024-09-21T14:24:08.572964+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:08.572964+0000","last_undegraded":"2024-09-21T14:24:08.572964+0000","last_fullsized":"2024-09-21T14:24:08.572964+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573806+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:08.573806+0000","last_peered":"2024-09-21T14:24:08.573806+0000","last_clean":"2024-09-21T14:24:08.573806+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:08.573806+0000","last_undegraded":"2024-09-21T14:24:08.573806+0000","last_fullsized":"2024-09-21T14:24:08.573806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572962+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:08.572962+0000","last_peered":"2024-09-21T14:24:08.572962+0000","last_clean":"2024-09-21T14:24:08.572962+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:08.572962+0000","last_undegraded":"2024-09-21T14:24:08.572962+0000","last_fullsized":"2024-09-21T14:24:08.572962+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6266,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573965+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:08.573965+0000","last_peered":"2024-09-21T14:24:08.573965+0000","last_clean":"2024-09-21T14:24:08.573965+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:08.573965+0000","last_undegraded":"2024-09-21T14:24:08.573965+0000","last_fullsized":"2024-09-21T14:24:08.573965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625986+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:07.625986+0000","last_peered":"2024-09-21T14:24:07.625986+0000","last_clean":"2024-09-21T14:24:07.625986+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:07.625986+0000","last_undegraded":"2024-09-21T14:24:07.625986+0000","last_fullsized":"2024-09-21T14:24:07.625986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3736,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626282+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:07.626282+0000","last_peered":"2024-09-21T14:24:07.626282+0000","last_clean":"2024-09-21T14:24:07.626282+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:07.626282+0000","last_undegraded":"2024-09-21T14:24:07.626282+0000","last_fullsized":"2024-09-21T14:24:07.626282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2983,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248820+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:07.248820+0000","last_peered":"2024-09-21T14:24:07.248820+0000","last_clean":"2024-09-21T14:24:07.248820+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:07.248820+0000","last_undegraded":"2024-09-21T14:24:07.248820+0000","last_fullsized":"2024-09-21T14:24:07.248820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:14.583949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625177+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:07.625177+0000","last_peered":"2024-09-21T14:24:07.625177+0000","last_clean":"2024-09-21T14:24:07.625177+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:07.625177+0000","last_undegraded":"2024-09-21T14:24:07.625177+0000","last_fullsized":"2024-09-21T14:24:07.625177+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:15.056601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259283+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:08.259283+0000","last_peered":"2024-09-21T14:24:08.259283+0000","last_clean":"2024-09-21T14:24:08.259283+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:08.259283+0000","last_undegraded":"2024-09-21T14:24:08.259283+0000","last_fullsized":"2024-09-21T14:24:08.259283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:34.912969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573015+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:08.573015+0000","last_peered":"2024-09-21T14:24:08.573015+0000","last_clean":"2024-09-21T14:24:08.573015+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:08.573015+0000","last_undegraded":"2024-09-21T14:24:08.573015+0000","last_fullsized":"2024-09-21T14:24:08.573015+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3306,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248798+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:07.248798+0000","last_peered":"2024-09-21T14:24:07.248798+0000","last_clean":"2024-09-21T14:24:07.248798+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:07.248798+0000","last_undegraded":"2024-09-21T14:24:07.248798+0000","last_fullsized":"2024-09-21T14:24:07.248798+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.949047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259426+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:08.259426+0000","last_peered":"2024-09-21T14:24:08.259426+0000","last_clean":"2024-09-21T14:24:08.259426+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:08.259426+0000","last_undegraded":"2024-09-21T14:24:08.259426+0000","last_fullsized":"2024-09-21T14:24:08.259426+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3446,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626177+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:07.626177+0000","last_peered":"2024-09-21T14:24:07.626177+0000","last_clean":"2024-09-21T14:24:07.626177+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:07.626177+0000","last_undegraded":"2024-09-21T14:24:07.626177+0000","last_fullsized":"2024-09-21T14:24:07.626177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248822+0000","last_change":"2024-09-21T14:23:49.431117+0000","last_active":"2024-09-21T14:24:07.248822+0000","last_peered":"2024-09-21T14:24:07.248822+0000","last_clean":"2024-09-21T14:24:07.248822+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:07.248822+0000","last_undegraded":"2024-09-21T14:24:07.248822+0000","last_fullsized":"2024-09-21T14:24:07.248822+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:12.106526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3322,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574105+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:08.574105+0000","last_peered":"2024-09-21T14:24:08.574105+0000","last_clean":"2024-09-21T14:24:08.574105+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:08.574105+0000","last_undegraded":"2024-09-21T14:24:08.574105+0000","last_fullsized":"2024-09-21T14:24:08.574105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":150,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625138+0000","last_change":"2024-09-21T14:23:56.392702+0000","last_active":"2024-09-21T14:24:07.625138+0000","last_peered":"2024-09-21T14:24:07.625138+0000","last_clean":"2024-09-21T14:24:07.625138+0000","last_became_active":"2024-09-21T14:23:56.392416+0000","last_became_peered":"2024-09-21T14:23:56.392416+0000","last_unstale":"2024-09-21T14:24:07.625138+0000","last_undegraded":"2024-09-21T14:24:07.625138+0000","last_fullsized":"2024-09-21T14:24:07.625138+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:58.517054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5776,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574132+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:08.574132+0000","last_peered":"2024-09-21T14:24:08.574132+0000","last_clean":"2024-09-21T14:24:08.574132+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:08.574132+0000","last_undegraded":"2024-09-21T14:24:08.574132+0000","last_fullsized":"2024-09-21T14:24:08.574132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625872+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:07.625872+0000","last_peered":"2024-09-21T14:24:07.625872+0000","last_clean":"2024-09-21T14:24:07.625872+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:07.625872+0000","last_undegraded":"2024-09-21T14:24:07.625872+0000","last_fullsized":"2024-09-21T14:24:07.625872+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":291,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.258998+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:08.258998+0000","last_peered":"2024-09-21T14:24:08.258998+0000","last_clean":"2024-09-21T14:24:08.258998+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:08.258998+0000","last_undegraded":"2024-09-21T14:24:08.258998+0000","last_fullsized":"2024-09-21T14:24:08.258998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":206,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573907+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:08.573907+0000","last_peered":"2024-09-21T14:24:08.573907+0000","last_clean":"2024-09-21T14:24:08.573907+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:08.573907+0000","last_undegraded":"2024-09-21T14:24:08.573907+0000","last_fullsized":"2024-09-21T14:24:08.573907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3525,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625976+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:07.625976+0000","last_peered":"2024-09-21T14:24:07.625976+0000","last_clean":"2024-09-21T14:24:07.625976+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:07.625976+0000","last_undegraded":"2024-09-21T14:24:07.625976+0000","last_fullsized":"2024-09-21T14:24:07.625976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.624926+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:07.624926+0000","last_peered":"2024-09-21T14:24:07.624926+0000","last_clean":"2024-09-21T14:24:07.624926+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:07.624926+0000","last_undegraded":"2024-09-21T14:24:07.624926+0000","last_fullsized":"2024-09-21T14:24:07.624926+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:46.419594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3006,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625807+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:07.625807+0000","last_peered":"2024-09-21T14:24:07.625807+0000","last_clean":"2024-09-21T14:24:07.625807+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:07.625807+0000","last_undegraded":"2024-09-21T14:24:07.625807+0000","last_fullsized":"2024-09-21T14:24:07.625807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626070+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:07.626070+0000","last_peered":"2024-09-21T14:24:07.626070+0000","last_clean":"2024-09-21T14:24:07.626070+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:07.626070+0000","last_undegraded":"2024-09-21T14:24:07.626070+0000","last_fullsized":"2024-09-21T14:24:07.626070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3053,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573765+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:08.573765+0000","last_peered":"2024-09-21T14:24:08.573765+0000","last_clean":"2024-09-21T14:24:08.573765+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:08.573765+0000","last_undegraded":"2024-09-21T14:24:08.573765+0000","last_fullsized":"2024-09-21T14:24:08.573765+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572963+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:08.572963+0000","last_peered":"2024-09-21T14:24:08.572963+0000","last_clean":"2024-09-21T14:24:08.572963+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:08.572963+0000","last_undegraded":"2024-09-21T14:24:08.572963+0000","last_fullsized":"2024-09-21T14:24:08.572963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3831,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573600+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:08.573600+0000","last_peered":"2024-09-21T14:24:08.573600+0000","last_clean":"2024-09-21T14:24:08.573600+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:08.573600+0000","last_undegraded":"2024-09-21T14:24:08.573600+0000","last_fullsized":"2024-09-21T14:24:08.573600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4809,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259111+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:08.259111+0000","last_peered":"2024-09-21T14:24:08.259111+0000","last_clean":"2024-09-21T14:24:08.259111+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:08.259111+0000","last_undegraded":"2024-09-21T14:24:08.259111+0000","last_fullsized":"2024-09-21T14:24:08.259111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259152+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:08.259152+0000","last_peered":"2024-09-21T14:24:08.259152+0000","last_clean":"2024-09-21T14:24:08.259152+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:08.259152+0000","last_undegraded":"2024-09-21T14:24:08.259152+0000","last_fullsized":"2024-09-21T14:24:08.259152+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2582,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626037+0000","last_change":"2024-09-21T14:23:46.158494+0000","last_active":"2024-09-21T14:24:07.626037+0000","last_peered":"2024-09-21T14:24:07.626037+0000","last_clean":"2024-09-21T14:24:07.626037+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:24:07.626037+0000","last_undegraded":"2024-09-21T14:24:07.626037+0000","last_fullsized":"2024-09-21T14:24:07.626037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894741,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2618,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625057+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:07.625057+0000","last_peered":"2024-09-21T14:24:07.625057+0000","last_clean":"2024-09-21T14:24:07.625057+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:07.625057+0000","last_undegraded":"2024-09-21T14:24:07.625057+0000","last_fullsized":"2024-09-21T14:24:07.625057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626196+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:07.626196+0000","last_peered":"2024-09-21T14:24:07.626196+0000","last_clean":"2024-09-21T14:24:07.626196+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:07.626196+0000","last_undegraded":"2024-09-21T14:24:07.626196+0000","last_fullsized":"2024-09-21T14:24:07.626196+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3911,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248806+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:07.248806+0000","last_peered":"2024-09-21T14:24:07.248806+0000","last_clean":"2024-09-21T14:24:07.248806+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:07.248806+0000","last_undegraded":"2024-09-21T14:24:07.248806+0000","last_fullsized":"2024-09-21T14:24:07.248806+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:32.841741+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625092+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:07.625092+0000","last_peered":"2024-09-21T14:24:07.625092+0000","last_clean":"2024-09-21T14:24:07.625092+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:07.625092+0000","last_undegraded":"2024-09-21T14:24:07.625092+0000","last_fullsized":"2024-09-21T14:24:07.625092+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:05.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3588,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572927+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:08.572927+0000","last_peered":"2024-09-21T14:24:08.572927+0000","last_clean":"2024-09-21T14:24:08.572927+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:08.572927+0000","last_undegraded":"2024-09-21T14:24:08.572927+0000","last_fullsized":"2024-09-21T14:24:08.572927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259197+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:08.259197+0000","last_peered":"2024-09-21T14:24:08.259197+0000","last_clean":"2024-09-21T14:24:08.259197+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:08.259197+0000","last_undegraded":"2024-09-21T14:24:08.259197+0000","last_fullsized":"2024-09-21T14:24:08.259197+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:21.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3775,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625902+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:07.625902+0000","last_peered":"2024-09-21T14:24:07.625902+0000","last_clean":"2024-09-21T14:24:07.625902+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:07.625902+0000","last_undegraded":"2024-09-21T14:24:07.625902+0000","last_fullsized":"2024-09-21T14:24:07.625902+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259038+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:08.259038+0000","last_peered":"2024-09-21T14:24:08.259038+0000","last_clean":"2024-09-21T14:24:08.259038+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:08.259038+0000","last_undegraded":"2024-09-21T14:24:08.259038+0000","last_fullsized":"2024-09-21T14:24:08.259038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.497}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2248704,"data_stored":1655571,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59548,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312292,"statfs":{"total":96636764160,"available":96575787008,"internally_reserved":0,"allocated":2244608,"data_stored":1662438,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.499},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.442}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29160,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342680,"statfs":{"total":96636764160,"available":96606904320,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.34399999999999997,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.44800000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.38500000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46899999999999997,"5min":0.621,"15min":0.621},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.44900000000000001,"15min":0.46899999999999997},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.55800000000000005,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.442},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":2.097,"15min":2.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":1071103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.251},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.308}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65252,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306588,"statfs":{"total":96636764160,"available":96569946112,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:12.955 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:24:12.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T14:24:13.242 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T14:24:13.254 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T14:24:13.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:13.526 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:13.526 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:13.538 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T14:24:12.447395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399012,"kb_used_data":17200,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575708,"statfs":{"total":773094113280,"available":772685524992,"internally_reserved":0,"allocated":17612800,"data_stored":9720000,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200102"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3711,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626213+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:07.626213+0000","last_peered":"2024-09-21T14:24:07.626213+0000","last_clean":"2024-09-21T14:24:07.626213+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:07.626213+0000","last_undegraded":"2024-09-21T14:24:07.626213+0000","last_fullsized":"2024-09-21T14:24:07.626213+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2728,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626140+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:07.626140+0000","last_peered":"2024-09-21T14:24:07.626140+0000","last_clean":"2024-09-21T14:24:07.626140+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:07.626140+0000","last_undegraded":"2024-09-21T14:24:07.626140+0000","last_fullsized":"2024-09-21T14:24:07.626140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3558,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259240+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:08.259240+0000","last_peered":"2024-09-21T14:24:08.259240+0000","last_clean":"2024-09-21T14:24:08.259240+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:08.259240+0000","last_undegraded":"2024-09-21T14:24:08.259240+0000","last_fullsized":"2024-09-21T14:24:08.259240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6356,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572964+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:08.572964+0000","last_peered":"2024-09-21T14:24:08.572964+0000","last_clean":"2024-09-21T14:24:08.572964+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:08.572964+0000","last_undegraded":"2024-09-21T14:24:08.572964+0000","last_fullsized":"2024-09-21T14:24:08.572964+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573806+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:08.573806+0000","last_peered":"2024-09-21T14:24:08.573806+0000","last_clean":"2024-09-21T14:24:08.573806+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:08.573806+0000","last_undegraded":"2024-09-21T14:24:08.573806+0000","last_fullsized":"2024-09-21T14:24:08.573806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572962+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:08.572962+0000","last_peered":"2024-09-21T14:24:08.572962+0000","last_clean":"2024-09-21T14:24:08.572962+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:08.572962+0000","last_undegraded":"2024-09-21T14:24:08.572962+0000","last_fullsized":"2024-09-21T14:24:08.572962+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6266,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573965+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:08.573965+0000","last_peered":"2024-09-21T14:24:08.573965+0000","last_clean":"2024-09-21T14:24:08.573965+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:08.573965+0000","last_undegraded":"2024-09-21T14:24:08.573965+0000","last_fullsized":"2024-09-21T14:24:08.573965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625986+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:07.625986+0000","last_peered":"2024-09-21T14:24:07.625986+0000","last_clean":"2024-09-21T14:24:07.625986+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:07.625986+0000","last_undegraded":"2024-09-21T14:24:07.625986+0000","last_fullsized":"2024-09-21T14:24:07.625986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3736,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626282+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:07.626282+0000","last_peered":"2024-09-21T14:24:07.626282+0000","last_clean":"2024-09-21T14:24:07.626282+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:07.626282+0000","last_undegraded":"2024-09-21T14:24:07.626282+0000","last_fullsized":"2024-09-21T14:24:07.626282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2983,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248820+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:07.248820+0000","last_peered":"2024-09-21T14:24:07.248820+0000","last_clean":"2024-09-21T14:24:07.248820+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:07.248820+0000","last_undegraded":"2024-09-21T14:24:07.248820+0000","last_fullsized":"2024-09-21T14:24:07.248820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:14.583949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625177+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:07.625177+0000","last_peered":"2024-09-21T14:24:07.625177+0000","last_clean":"2024-09-21T14:24:07.625177+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:07.625177+0000","last_undegraded":"2024-09-21T14:24:07.625177+0000","last_fullsized":"2024-09-21T14:24:07.625177+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:15.056601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259283+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:08.259283+0000","last_peered":"2024-09-21T14:24:08.259283+0000","last_clean":"2024-09-21T14:24:08.259283+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:08.259283+0000","last_undegraded":"2024-09-21T14:24:08.259283+0000","last_fullsized":"2024-09-21T14:24:08.259283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:34.912969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573015+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:08.573015+0000","last_peered":"2024-09-21T14:24:08.573015+0000","last_clean":"2024-09-21T14:24:08.573015+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:08.573015+0000","last_undegraded":"2024-09-21T14:24:08.573015+0000","last_fullsized":"2024-09-21T14:24:08.573015+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3306,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248798+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:07.248798+0000","last_peered":"2024-09-21T14:24:07.248798+0000","last_clean":"2024-09-21T14:24:07.248798+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:07.248798+0000","last_undegraded":"2024-09-21T14:24:07.248798+0000","last_fullsized":"2024-09-21T14:24:07.248798+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.949047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259426+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:08.259426+0000","last_peered":"2024-09-21T14:24:08.259426+0000","last_clean":"2024-09-21T14:24:08.259426+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:08.259426+0000","last_undegraded":"2024-09-21T14:24:08.259426+0000","last_fullsized":"2024-09-21T14:24:08.259426+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3446,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626177+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:07.626177+0000","last_peered":"2024-09-21T14:24:07.626177+0000","last_clean":"2024-09-21T14:24:07.626177+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:07.626177+0000","last_undegraded":"2024-09-21T14:24:07.626177+0000","last_fullsized":"2024-09-21T14:24:07.626177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248822+0000","last_change":"2024-09-21T14:23:49.431117+0000","last_active":"2024-09-21T14:24:07.248822+0000","last_peered":"2024-09-21T14:24:07.248822+0000","last_clean":"2024-09-21T14:24:07.248822+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:07.248822+0000","last_undegraded":"2024-09-21T14:24:07.248822+0000","last_fullsized":"2024-09-21T14:24:07.248822+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:12.106526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3322,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574105+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:08.574105+0000","last_peered":"2024-09-21T14:24:08.574105+0000","last_clean":"2024-09-21T14:24:08.574105+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:08.574105+0000","last_undegraded":"2024-09-21T14:24:08.574105+0000","last_fullsized":"2024-09-21T14:24:08.574105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":150,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625138+0000","last_change":"2024-09-21T14:23:56.392702+0000","last_active":"2024-09-21T14:24:07.625138+0000","last_peered":"2024-09-21T14:24:07.625138+0000","last_clean":"2024-09-21T14:24:07.625138+0000","last_became_active":"2024-09-21T14:23:56.392416+0000","last_became_peered":"2024-09-21T14:23:56.392416+0000","last_unstale":"2024-09-21T14:24:07.625138+0000","last_undegraded":"2024-09-21T14:24:07.625138+0000","last_fullsized":"2024-09-21T14:24:07.625138+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:58.517054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5776,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574132+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:08.574132+0000","last_peered":"2024-09-21T14:24:08.574132+0000","last_clean":"2024-09-21T14:24:08.574132+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:08.574132+0000","last_undegraded":"2024-09-21T14:24:08.574132+0000","last_fullsized":"2024-09-21T14:24:08.574132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625872+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:07.625872+0000","last_peered":"2024-09-21T14:24:07.625872+0000","last_clean":"2024-09-21T14:24:07.625872+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:07.625872+0000","last_undegraded":"2024-09-21T14:24:07.625872+0000","last_fullsized":"2024-09-21T14:24:07.625872+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":291,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.258998+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:08.258998+0000","last_peered":"2024-09-21T14:24:08.258998+0000","last_clean":"2024-09-21T14:24:08.258998+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:08.258998+0000","last_undegraded":"2024-09-21T14:24:08.258998+0000","last_fullsized":"2024-09-21T14:24:08.258998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":206,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573907+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:08.573907+0000","last_peered":"2024-09-21T14:24:08.573907+0000","last_clean":"2024-09-21T14:24:08.573907+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:08.573907+0000","last_undegraded":"2024-09-21T14:24:08.573907+0000","last_fullsized":"2024-09-21T14:24:08.573907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3525,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625976+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:07.625976+0000","last_peered":"2024-09-21T14:24:07.625976+0000","last_clean":"2024-09-21T14:24:07.625976+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:07.625976+0000","last_undegraded":"2024-09-21T14:24:07.625976+0000","last_fullsized":"2024-09-21T14:24:07.625976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.624926+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:07.624926+0000","last_peered":"2024-09-21T14:24:07.624926+0000","last_clean":"2024-09-21T14:24:07.624926+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:07.624926+0000","last_undegraded":"2024-09-21T14:24:07.624926+0000","last_fullsized":"2024-09-21T14:24:07.624926+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:46.419594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3006,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625807+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:07.625807+0000","last_peered":"2024-09-21T14:24:07.625807+0000","last_clean":"2024-09-21T14:24:07.625807+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:07.625807+0000","last_undegraded":"2024-09-21T14:24:07.625807+0000","last_fullsized":"2024-09-21T14:24:07.625807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626070+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:07.626070+0000","last_peered":"2024-09-21T14:24:07.626070+0000","last_clean":"2024-09-21T14:24:07.626070+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:07.626070+0000","last_undegraded":"2024-09-21T14:24:07.626070+0000","last_fullsized":"2024-09-21T14:24:07.626070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3053,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573765+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:08.573765+0000","last_peered":"2024-09-21T14:24:08.573765+0000","last_clean":"2024-09-21T14:24:08.573765+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:08.573765+0000","last_undegraded":"2024-09-21T14:24:08.573765+0000","last_fullsized":"2024-09-21T14:24:08.573765+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572963+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:08.572963+0000","last_peered":"2024-09-21T14:24:08.572963+0000","last_clean":"2024-09-21T14:24:08.572963+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:08.572963+0000","last_undegraded":"2024-09-21T14:24:08.572963+0000","last_fullsized":"2024-09-21T14:24:08.572963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3831,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573600+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:08.573600+0000","last_peered":"2024-09-21T14:24:08.573600+0000","last_clean":"2024-09-21T14:24:08.573600+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:08.573600+0000","last_undegraded":"2024-09-21T14:24:08.573600+0000","last_fullsized":"2024-09-21T14:24:08.573600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4809,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259111+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:08.259111+0000","last_peered":"2024-09-21T14:24:08.259111+0000","last_clean":"2024-09-21T14:24:08.259111+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:08.259111+0000","last_undegraded":"2024-09-21T14:24:08.259111+0000","last_fullsized":"2024-09-21T14:24:08.259111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259152+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:08.259152+0000","last_peered":"2024-09-21T14:24:08.259152+0000","last_clean":"2024-09-21T14:24:08.259152+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:08.259152+0000","last_undegraded":"2024-09-21T14:24:08.259152+0000","last_fullsized":"2024-09-21T14:24:08.259152+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2582,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626037+0000","last_change":"2024-09-21T14:23:46.158494+0000","last_active":"2024-09-21T14:24:07.626037+0000","last_peered":"2024-09-21T14:24:07.626037+0000","last_clean":"2024-09-21T14:24:07.626037+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:24:07.626037+0000","last_undegraded":"2024-09-21T14:24:07.626037+0000","last_fullsized":"2024-09-21T14:24:07.626037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894741,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2618,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625057+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:07.625057+0000","last_peered":"2024-09-21T14:24:07.625057+0000","last_clean":"2024-09-21T14:24:07.625057+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:07.625057+0000","last_undegraded":"2024-09-21T14:24:07.625057+0000","last_fullsized":"2024-09-21T14:24:07.625057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626196+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:07.626196+0000","last_peered":"2024-09-21T14:24:07.626196+0000","last_clean":"2024-09-21T14:24:07.626196+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:07.626196+0000","last_undegraded":"2024-09-21T14:24:07.626196+0000","last_fullsized":"2024-09-21T14:24:07.626196+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3911,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248806+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:07.248806+0000","last_peered":"2024-09-21T14:24:07.248806+0000","last_clean":"2024-09-21T14:24:07.248806+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:07.248806+0000","last_undegraded":"2024-09-21T14:24:07.248806+0000","last_fullsized":"2024-09-21T14:24:07.248806+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:32.841741+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625092+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:07.625092+0000","last_peered":"2024-09-21T14:24:07.625092+0000","last_clean":"2024-09-21T14:24:07.625092+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:07.625092+0000","last_undegraded":"2024-09-21T14:24:07.625092+0000","last_fullsized":"2024-09-21T14:24:07.625092+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:05.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3588,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572927+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:08.572927+0000","last_peered":"2024-09-21T14:24:08.572927+0000","last_clean":"2024-09-21T14:24:08.572927+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:08.572927+0000","last_undegraded":"2024-09-21T14:24:08.572927+0000","last_fullsized":"2024-09-21T14:24:08.572927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259197+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:08.259197+0000","last_peered":"2024-09-21T14:24:08.259197+0000","last_clean":"2024-09-21T14:24:08.259197+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:08.259197+0000","last_undegraded":"2024-09-21T14:24:08.259197+0000","last_fullsized":"2024-09-21T14:24:08.259197+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:21.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3775,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625902+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:07.625902+0000","last_peered":"2024-09-21T14:24:07.625902+0000","last_clean":"2024-09-21T14:24:07.625902+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:07.625902+0000","last_undegraded":"2024-09-21T14:24:07.625902+0000","last_fullsized":"2024-09-21T14:24:07.625902+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259038+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:08.259038+0000","last_peered":"2024-09-21T14:24:08.259038+0000","last_clean":"2024-09-21T14:24:08.259038+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:08.259038+0000","last_undegraded":"2024-09-21T14:24:08.259038+0000","last_fullsized":"2024-09-21T14:24:08.259038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.497}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2248704,"data_stored":1655571,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59548,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312292,"statfs":{"total":96636764160,"available":96575787008,"internally_reserved":0,"allocated":2244608,"data_stored":1662438,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.499},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.442}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29160,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342680,"statfs":{"total":96636764160,"available":96606904320,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.34399999999999997,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.44800000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.38500000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46899999999999997,"5min":0.621,"15min":0.621},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.44900000000000001,"15min":0.46899999999999997},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.55800000000000005,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.442},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":2.097,"15min":2.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":1071103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.251},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.308}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65252,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306588,"statfs":{"total":96636764160,"available":96569946112,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:13.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:13.812 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:13.813 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:13.823 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T14:24:12.447395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399012,"kb_used_data":17200,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575708,"statfs":{"total":773094113280,"available":772685524992,"internally_reserved":0,"allocated":17612800,"data_stored":9720000,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200102"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3711,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626213+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:07.626213+0000","last_peered":"2024-09-21T14:24:07.626213+0000","last_clean":"2024-09-21T14:24:07.626213+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:07.626213+0000","last_undegraded":"2024-09-21T14:24:07.626213+0000","last_fullsized":"2024-09-21T14:24:07.626213+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2728,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626140+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:07.626140+0000","last_peered":"2024-09-21T14:24:07.626140+0000","last_clean":"2024-09-21T14:24:07.626140+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:07.626140+0000","last_undegraded":"2024-09-21T14:24:07.626140+0000","last_fullsized":"2024-09-21T14:24:07.626140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3558,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259240+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:08.259240+0000","last_peered":"2024-09-21T14:24:08.259240+0000","last_clean":"2024-09-21T14:24:08.259240+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:08.259240+0000","last_undegraded":"2024-09-21T14:24:08.259240+0000","last_fullsized":"2024-09-21T14:24:08.259240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6356,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572964+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:08.572964+0000","last_peered":"2024-09-21T14:24:08.572964+0000","last_clean":"2024-09-21T14:24:08.572964+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:08.572964+0000","last_undegraded":"2024-09-21T14:24:08.572964+0000","last_fullsized":"2024-09-21T14:24:08.572964+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573806+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:08.573806+0000","last_peered":"2024-09-21T14:24:08.573806+0000","last_clean":"2024-09-21T14:24:08.573806+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:08.573806+0000","last_undegraded":"2024-09-21T14:24:08.573806+0000","last_fullsized":"2024-09-21T14:24:08.573806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572962+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:08.572962+0000","last_peered":"2024-09-21T14:24:08.572962+0000","last_clean":"2024-09-21T14:24:08.572962+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:08.572962+0000","last_undegraded":"2024-09-21T14:24:08.572962+0000","last_fullsized":"2024-09-21T14:24:08.572962+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6266,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573965+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:08.573965+0000","last_peered":"2024-09-21T14:24:08.573965+0000","last_clean":"2024-09-21T14:24:08.573965+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:08.573965+0000","last_undegraded":"2024-09-21T14:24:08.573965+0000","last_fullsized":"2024-09-21T14:24:08.573965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625986+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:07.625986+0000","last_peered":"2024-09-21T14:24:07.625986+0000","last_clean":"2024-09-21T14:24:07.625986+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:07.625986+0000","last_undegraded":"2024-09-21T14:24:07.625986+0000","last_fullsized":"2024-09-21T14:24:07.625986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3736,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626282+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:07.626282+0000","last_peered":"2024-09-21T14:24:07.626282+0000","last_clean":"2024-09-21T14:24:07.626282+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:07.626282+0000","last_undegraded":"2024-09-21T14:24:07.626282+0000","last_fullsized":"2024-09-21T14:24:07.626282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2983,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248820+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:07.248820+0000","last_peered":"2024-09-21T14:24:07.248820+0000","last_clean":"2024-09-21T14:24:07.248820+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:07.248820+0000","last_undegraded":"2024-09-21T14:24:07.248820+0000","last_fullsized":"2024-09-21T14:24:07.248820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:14.583949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625177+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:07.625177+0000","last_peered":"2024-09-21T14:24:07.625177+0000","last_clean":"2024-09-21T14:24:07.625177+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:07.625177+0000","last_undegraded":"2024-09-21T14:24:07.625177+0000","last_fullsized":"2024-09-21T14:24:07.625177+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:15.056601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259283+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:08.259283+0000","last_peered":"2024-09-21T14:24:08.259283+0000","last_clean":"2024-09-21T14:24:08.259283+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:08.259283+0000","last_undegraded":"2024-09-21T14:24:08.259283+0000","last_fullsized":"2024-09-21T14:24:08.259283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:34.912969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573015+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:08.573015+0000","last_peered":"2024-09-21T14:24:08.573015+0000","last_clean":"2024-09-21T14:24:08.573015+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:08.573015+0000","last_undegraded":"2024-09-21T14:24:08.573015+0000","last_fullsized":"2024-09-21T14:24:08.573015+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3306,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248798+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:07.248798+0000","last_peered":"2024-09-21T14:24:07.248798+0000","last_clean":"2024-09-21T14:24:07.248798+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:07.248798+0000","last_undegraded":"2024-09-21T14:24:07.248798+0000","last_fullsized":"2024-09-21T14:24:07.248798+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.949047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259426+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:08.259426+0000","last_peered":"2024-09-21T14:24:08.259426+0000","last_clean":"2024-09-21T14:24:08.259426+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:08.259426+0000","last_undegraded":"2024-09-21T14:24:08.259426+0000","last_fullsized":"2024-09-21T14:24:08.259426+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3446,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626177+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:07.626177+0000","last_peered":"2024-09-21T14:24:07.626177+0000","last_clean":"2024-09-21T14:24:07.626177+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:07.626177+0000","last_undegraded":"2024-09-21T14:24:07.626177+0000","last_fullsized":"2024-09-21T14:24:07.626177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248822+0000","last_change":"2024-09-21T14:23:49.431117+0000","last_active":"2024-09-21T14:24:07.248822+0000","last_peered":"2024-09-21T14:24:07.248822+0000","last_clean":"2024-09-21T14:24:07.248822+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:07.248822+0000","last_undegraded":"2024-09-21T14:24:07.248822+0000","last_fullsized":"2024-09-21T14:24:07.248822+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:12.106526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3322,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574105+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:08.574105+0000","last_peered":"2024-09-21T14:24:08.574105+0000","last_clean":"2024-09-21T14:24:08.574105+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:08.574105+0000","last_undegraded":"2024-09-21T14:24:08.574105+0000","last_fullsized":"2024-09-21T14:24:08.574105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":150,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625138+0000","last_change":"2024-09-21T14:23:56.392702+0000","last_active":"2024-09-21T14:24:07.625138+0000","last_peered":"2024-09-21T14:24:07.625138+0000","last_clean":"2024-09-21T14:24:07.625138+0000","last_became_active":"2024-09-21T14:23:56.392416+0000","last_became_peered":"2024-09-21T14:23:56.392416+0000","last_unstale":"2024-09-21T14:24:07.625138+0000","last_undegraded":"2024-09-21T14:24:07.625138+0000","last_fullsized":"2024-09-21T14:24:07.625138+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:58.517054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5776,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574132+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:08.574132+0000","last_peered":"2024-09-21T14:24:08.574132+0000","last_clean":"2024-09-21T14:24:08.574132+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:08.574132+0000","last_undegraded":"2024-09-21T14:24:08.574132+0000","last_fullsized":"2024-09-21T14:24:08.574132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625872+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:07.625872+0000","last_peered":"2024-09-21T14:24:07.625872+0000","last_clean":"2024-09-21T14:24:07.625872+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:07.625872+0000","last_undegraded":"2024-09-21T14:24:07.625872+0000","last_fullsized":"2024-09-21T14:24:07.625872+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":291,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.258998+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:08.258998+0000","last_peered":"2024-09-21T14:24:08.258998+0000","last_clean":"2024-09-21T14:24:08.258998+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:08.258998+0000","last_undegraded":"2024-09-21T14:24:08.258998+0000","last_fullsized":"2024-09-21T14:24:08.258998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":206,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573907+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:08.573907+0000","last_peered":"2024-09-21T14:24:08.573907+0000","last_clean":"2024-09-21T14:24:08.573907+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:08.573907+0000","last_undegraded":"2024-09-21T14:24:08.573907+0000","last_fullsized":"2024-09-21T14:24:08.573907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3525,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625976+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:07.625976+0000","last_peered":"2024-09-21T14:24:07.625976+0000","last_clean":"2024-09-21T14:24:07.625976+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:07.625976+0000","last_undegraded":"2024-09-21T14:24:07.625976+0000","last_fullsized":"2024-09-21T14:24:07.625976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.624926+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:07.624926+0000","last_peered":"2024-09-21T14:24:07.624926+0000","last_clean":"2024-09-21T14:24:07.624926+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:07.624926+0000","last_undegraded":"2024-09-21T14:24:07.624926+0000","last_fullsized":"2024-09-21T14:24:07.624926+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:46.419594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3006,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625807+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:07.625807+0000","last_peered":"2024-09-21T14:24:07.625807+0000","last_clean":"2024-09-21T14:24:07.625807+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:07.625807+0000","last_undegraded":"2024-09-21T14:24:07.625807+0000","last_fullsized":"2024-09-21T14:24:07.625807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626070+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:07.626070+0000","last_peered":"2024-09-21T14:24:07.626070+0000","last_clean":"2024-09-21T14:24:07.626070+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:07.626070+0000","last_undegraded":"2024-09-21T14:24:07.626070+0000","last_fullsized":"2024-09-21T14:24:07.626070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3053,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573765+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:08.573765+0000","last_peered":"2024-09-21T14:24:08.573765+0000","last_clean":"2024-09-21T14:24:08.573765+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:08.573765+0000","last_undegraded":"2024-09-21T14:24:08.573765+0000","last_fullsized":"2024-09-21T14:24:08.573765+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572963+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:08.572963+0000","last_peered":"2024-09-21T14:24:08.572963+0000","last_clean":"2024-09-21T14:24:08.572963+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:08.572963+0000","last_undegraded":"2024-09-21T14:24:08.572963+0000","last_fullsized":"2024-09-21T14:24:08.572963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3831,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573600+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:08.573600+0000","last_peered":"2024-09-21T14:24:08.573600+0000","last_clean":"2024-09-21T14:24:08.573600+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:08.573600+0000","last_undegraded":"2024-09-21T14:24:08.573600+0000","last_fullsized":"2024-09-21T14:24:08.573600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4809,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259111+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:08.259111+0000","last_peered":"2024-09-21T14:24:08.259111+0000","last_clean":"2024-09-21T14:24:08.259111+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:08.259111+0000","last_undegraded":"2024-09-21T14:24:08.259111+0000","last_fullsized":"2024-09-21T14:24:08.259111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259152+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:08.259152+0000","last_peered":"2024-09-21T14:24:08.259152+0000","last_clean":"2024-09-21T14:24:08.259152+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:08.259152+0000","last_undegraded":"2024-09-21T14:24:08.259152+0000","last_fullsized":"2024-09-21T14:24:08.259152+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2582,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626037+0000","last_change":"2024-09-21T14:23:46.158494+0000","last_active":"2024-09-21T14:24:07.626037+0000","last_peered":"2024-09-21T14:24:07.626037+0000","last_clean":"2024-09-21T14:24:07.626037+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:24:07.626037+0000","last_undegraded":"2024-09-21T14:24:07.626037+0000","last_fullsized":"2024-09-21T14:24:07.626037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894741,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2618,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625057+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:07.625057+0000","last_peered":"2024-09-21T14:24:07.625057+0000","last_clean":"2024-09-21T14:24:07.625057+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:07.625057+0000","last_undegraded":"2024-09-21T14:24:07.625057+0000","last_fullsized":"2024-09-21T14:24:07.625057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626196+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:07.626196+0000","last_peered":"2024-09-21T14:24:07.626196+0000","last_clean":"2024-09-21T14:24:07.626196+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:07.626196+0000","last_undegraded":"2024-09-21T14:24:07.626196+0000","last_fullsized":"2024-09-21T14:24:07.626196+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3911,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248806+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:07.248806+0000","last_peered":"2024-09-21T14:24:07.248806+0000","last_clean":"2024-09-21T14:24:07.248806+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:07.248806+0000","last_undegraded":"2024-09-21T14:24:07.248806+0000","last_fullsized":"2024-09-21T14:24:07.248806+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:32.841741+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625092+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:07.625092+0000","last_peered":"2024-09-21T14:24:07.625092+0000","last_clean":"2024-09-21T14:24:07.625092+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:07.625092+0000","last_undegraded":"2024-09-21T14:24:07.625092+0000","last_fullsized":"2024-09-21T14:24:07.625092+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:05.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3588,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572927+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:08.572927+0000","last_peered":"2024-09-21T14:24:08.572927+0000","last_clean":"2024-09-21T14:24:08.572927+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:08.572927+0000","last_undegraded":"2024-09-21T14:24:08.572927+0000","last_fullsized":"2024-09-21T14:24:08.572927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259197+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:08.259197+0000","last_peered":"2024-09-21T14:24:08.259197+0000","last_clean":"2024-09-21T14:24:08.259197+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:08.259197+0000","last_undegraded":"2024-09-21T14:24:08.259197+0000","last_fullsized":"2024-09-21T14:24:08.259197+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:21.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3775,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625902+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:07.625902+0000","last_peered":"2024-09-21T14:24:07.625902+0000","last_clean":"2024-09-21T14:24:07.625902+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:07.625902+0000","last_undegraded":"2024-09-21T14:24:07.625902+0000","last_fullsized":"2024-09-21T14:24:07.625902+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259038+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:08.259038+0000","last_peered":"2024-09-21T14:24:08.259038+0000","last_clean":"2024-09-21T14:24:08.259038+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:08.259038+0000","last_undegraded":"2024-09-21T14:24:08.259038+0000","last_fullsized":"2024-09-21T14:24:08.259038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.497}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2248704,"data_stored":1655571,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59548,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312292,"statfs":{"total":96636764160,"available":96575787008,"internally_reserved":0,"allocated":2244608,"data_stored":1662438,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.499},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.442}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29160,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342680,"statfs":{"total":96636764160,"available":96606904320,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.34399999999999997,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.44800000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.38500000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46899999999999997,"5min":0.621,"15min":0.621},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.44900000000000001,"15min":0.46899999999999997},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.55800000000000005,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.442},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":2.097,"15min":2.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":1071103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.251},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.308}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65252,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306588,"statfs":{"total":96636764160,"available":96569946112,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:13.826 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T14:24:13.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:14.097 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:14.098 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:14.109 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T14:24:12.447395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399012,"kb_used_data":17200,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575708,"statfs":{"total":773094113280,"available":772685524992,"internally_reserved":0,"allocated":17612800,"data_stored":9720000,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.200102"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3711,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626213+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:07.626213+0000","last_peered":"2024-09-21T14:24:07.626213+0000","last_clean":"2024-09-21T14:24:07.626213+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:07.626213+0000","last_undegraded":"2024-09-21T14:24:07.626213+0000","last_fullsized":"2024-09-21T14:24:07.626213+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2728,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626140+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:07.626140+0000","last_peered":"2024-09-21T14:24:07.626140+0000","last_clean":"2024-09-21T14:24:07.626140+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:07.626140+0000","last_undegraded":"2024-09-21T14:24:07.626140+0000","last_fullsized":"2024-09-21T14:24:07.626140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3558,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259240+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:08.259240+0000","last_peered":"2024-09-21T14:24:08.259240+0000","last_clean":"2024-09-21T14:24:08.259240+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:08.259240+0000","last_undegraded":"2024-09-21T14:24:08.259240+0000","last_fullsized":"2024-09-21T14:24:08.259240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:25.292530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6356,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572964+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:08.572964+0000","last_peered":"2024-09-21T14:24:08.572964+0000","last_clean":"2024-09-21T14:24:08.572964+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:08.572964+0000","last_undegraded":"2024-09-21T14:24:08.572964+0000","last_fullsized":"2024-09-21T14:24:08.572964+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:52.174181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573806+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:08.573806+0000","last_peered":"2024-09-21T14:24:08.573806+0000","last_clean":"2024-09-21T14:24:08.573806+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:08.573806+0000","last_undegraded":"2024-09-21T14:24:08.573806+0000","last_fullsized":"2024-09-21T14:24:08.573806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572962+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:08.572962+0000","last_peered":"2024-09-21T14:24:08.572962+0000","last_clean":"2024-09-21T14:24:08.572962+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:08.572962+0000","last_undegraded":"2024-09-21T14:24:08.572962+0000","last_fullsized":"2024-09-21T14:24:08.572962+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:48.704950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6266,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573965+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:08.573965+0000","last_peered":"2024-09-21T14:24:08.573965+0000","last_clean":"2024-09-21T14:24:08.573965+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:08.573965+0000","last_undegraded":"2024-09-21T14:24:08.573965+0000","last_fullsized":"2024-09-21T14:24:08.573965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625986+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:07.625986+0000","last_peered":"2024-09-21T14:24:07.625986+0000","last_clean":"2024-09-21T14:24:07.625986+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:07.625986+0000","last_undegraded":"2024-09-21T14:24:07.625986+0000","last_fullsized":"2024-09-21T14:24:07.625986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3736,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626282+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:07.626282+0000","last_peered":"2024-09-21T14:24:07.626282+0000","last_clean":"2024-09-21T14:24:07.626282+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:07.626282+0000","last_undegraded":"2024-09-21T14:24:07.626282+0000","last_fullsized":"2024-09-21T14:24:07.626282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2983,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248820+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:07.248820+0000","last_peered":"2024-09-21T14:24:07.248820+0000","last_clean":"2024-09-21T14:24:07.248820+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:07.248820+0000","last_undegraded":"2024-09-21T14:24:07.248820+0000","last_fullsized":"2024-09-21T14:24:07.248820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:14.583949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625177+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:07.625177+0000","last_peered":"2024-09-21T14:24:07.625177+0000","last_clean":"2024-09-21T14:24:07.625177+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:07.625177+0000","last_undegraded":"2024-09-21T14:24:07.625177+0000","last_fullsized":"2024-09-21T14:24:07.625177+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:15.056601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259283+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:08.259283+0000","last_peered":"2024-09-21T14:24:08.259283+0000","last_clean":"2024-09-21T14:24:08.259283+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:08.259283+0000","last_undegraded":"2024-09-21T14:24:08.259283+0000","last_fullsized":"2024-09-21T14:24:08.259283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:34.912969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573015+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:08.573015+0000","last_peered":"2024-09-21T14:24:08.573015+0000","last_clean":"2024-09-21T14:24:08.573015+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:08.573015+0000","last_undegraded":"2024-09-21T14:24:08.573015+0000","last_fullsized":"2024-09-21T14:24:08.573015+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:25.511931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3306,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248798+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:07.248798+0000","last_peered":"2024-09-21T14:24:07.248798+0000","last_clean":"2024-09-21T14:24:07.248798+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:07.248798+0000","last_undegraded":"2024-09-21T14:24:07.248798+0000","last_fullsized":"2024-09-21T14:24:07.248798+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.949047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259426+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:08.259426+0000","last_peered":"2024-09-21T14:24:08.259426+0000","last_clean":"2024-09-21T14:24:08.259426+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:08.259426+0000","last_undegraded":"2024-09-21T14:24:08.259426+0000","last_fullsized":"2024-09-21T14:24:08.259426+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.108265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3446,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626177+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:07.626177+0000","last_peered":"2024-09-21T14:24:07.626177+0000","last_clean":"2024-09-21T14:24:07.626177+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:07.626177+0000","last_undegraded":"2024-09-21T14:24:07.626177+0000","last_fullsized":"2024-09-21T14:24:07.626177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248822+0000","last_change":"2024-09-21T14:23:49.431117+0000","last_active":"2024-09-21T14:24:07.248822+0000","last_peered":"2024-09-21T14:24:07.248822+0000","last_clean":"2024-09-21T14:24:07.248822+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:07.248822+0000","last_undegraded":"2024-09-21T14:24:07.248822+0000","last_fullsized":"2024-09-21T14:24:07.248822+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:12.106526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3322,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574105+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:08.574105+0000","last_peered":"2024-09-21T14:24:08.574105+0000","last_clean":"2024-09-21T14:24:08.574105+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:08.574105+0000","last_undegraded":"2024-09-21T14:24:08.574105+0000","last_fullsized":"2024-09-21T14:24:08.574105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":150,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625138+0000","last_change":"2024-09-21T14:23:56.392702+0000","last_active":"2024-09-21T14:24:07.625138+0000","last_peered":"2024-09-21T14:24:07.625138+0000","last_clean":"2024-09-21T14:24:07.625138+0000","last_became_active":"2024-09-21T14:23:56.392416+0000","last_became_peered":"2024-09-21T14:23:56.392416+0000","last_unstale":"2024-09-21T14:24:07.625138+0000","last_undegraded":"2024-09-21T14:24:07.625138+0000","last_fullsized":"2024-09-21T14:24:07.625138+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:48:58.517054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5776,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.574132+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:08.574132+0000","last_peered":"2024-09-21T14:24:08.574132+0000","last_clean":"2024-09-21T14:24:08.574132+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:08.574132+0000","last_undegraded":"2024-09-21T14:24:08.574132+0000","last_fullsized":"2024-09-21T14:24:08.574132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625872+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:07.625872+0000","last_peered":"2024-09-21T14:24:07.625872+0000","last_clean":"2024-09-21T14:24:07.625872+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:07.625872+0000","last_undegraded":"2024-09-21T14:24:07.625872+0000","last_fullsized":"2024-09-21T14:24:07.625872+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":291,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.258998+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:08.258998+0000","last_peered":"2024-09-21T14:24:08.258998+0000","last_clean":"2024-09-21T14:24:08.258998+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:08.258998+0000","last_undegraded":"2024-09-21T14:24:08.258998+0000","last_fullsized":"2024-09-21T14:24:08.258998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:43.978107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":206,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573907+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:08.573907+0000","last_peered":"2024-09-21T14:24:08.573907+0000","last_clean":"2024-09-21T14:24:08.573907+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:08.573907+0000","last_undegraded":"2024-09-21T14:24:08.573907+0000","last_fullsized":"2024-09-21T14:24:08.573907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3525,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625976+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:07.625976+0000","last_peered":"2024-09-21T14:24:07.625976+0000","last_clean":"2024-09-21T14:24:07.625976+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:07.625976+0000","last_undegraded":"2024-09-21T14:24:07.625976+0000","last_fullsized":"2024-09-21T14:24:07.625976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.624926+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:07.624926+0000","last_peered":"2024-09-21T14:24:07.624926+0000","last_clean":"2024-09-21T14:24:07.624926+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:07.624926+0000","last_undegraded":"2024-09-21T14:24:07.624926+0000","last_fullsized":"2024-09-21T14:24:07.624926+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:46.419594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3006,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625807+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:07.625807+0000","last_peered":"2024-09-21T14:24:07.625807+0000","last_clean":"2024-09-21T14:24:07.625807+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:07.625807+0000","last_undegraded":"2024-09-21T14:24:07.625807+0000","last_fullsized":"2024-09-21T14:24:07.625807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626070+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:07.626070+0000","last_peered":"2024-09-21T14:24:07.626070+0000","last_clean":"2024-09-21T14:24:07.626070+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:07.626070+0000","last_undegraded":"2024-09-21T14:24:07.626070+0000","last_fullsized":"2024-09-21T14:24:07.626070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3053,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573765+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:08.573765+0000","last_peered":"2024-09-21T14:24:08.573765+0000","last_clean":"2024-09-21T14:24:08.573765+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:08.573765+0000","last_undegraded":"2024-09-21T14:24:08.573765+0000","last_fullsized":"2024-09-21T14:24:08.573765+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572963+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:08.572963+0000","last_peered":"2024-09-21T14:24:08.572963+0000","last_clean":"2024-09-21T14:24:08.572963+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:08.572963+0000","last_undegraded":"2024-09-21T14:24:08.572963+0000","last_fullsized":"2024-09-21T14:24:08.572963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:54:21.696214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3831,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.573600+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:08.573600+0000","last_peered":"2024-09-21T14:24:08.573600+0000","last_clean":"2024-09-21T14:24:08.573600+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:08.573600+0000","last_undegraded":"2024-09-21T14:24:08.573600+0000","last_fullsized":"2024-09-21T14:24:08.573600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4809,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259111+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:08.259111+0000","last_peered":"2024-09-21T14:24:08.259111+0000","last_clean":"2024-09-21T14:24:08.259111+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:08.259111+0000","last_undegraded":"2024-09-21T14:24:08.259111+0000","last_fullsized":"2024-09-21T14:24:08.259111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259152+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:08.259152+0000","last_peered":"2024-09-21T14:24:08.259152+0000","last_clean":"2024-09-21T14:24:08.259152+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:08.259152+0000","last_undegraded":"2024-09-21T14:24:08.259152+0000","last_fullsized":"2024-09-21T14:24:08.259152+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:49.690066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2582,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626037+0000","last_change":"2024-09-21T14:23:46.158494+0000","last_active":"2024-09-21T14:24:07.626037+0000","last_peered":"2024-09-21T14:24:07.626037+0000","last_clean":"2024-09-21T14:24:07.626037+0000","last_became_active":"2024-09-21T14:23:18.269117+0000","last_became_peered":"2024-09-21T14:23:18.269117+0000","last_unstale":"2024-09-21T14:24:07.626037+0000","last_undegraded":"2024-09-21T14:24:07.626037+0000","last_fullsized":"2024-09-21T14:24:07.626037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:38.969031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894741,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2618,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625057+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:07.625057+0000","last_peered":"2024-09-21T14:24:07.625057+0000","last_clean":"2024-09-21T14:24:07.625057+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:07.625057+0000","last_undegraded":"2024-09-21T14:24:07.625057+0000","last_fullsized":"2024-09-21T14:24:07.625057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:48.747479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3094,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.626196+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:07.626196+0000","last_peered":"2024-09-21T14:24:07.626196+0000","last_clean":"2024-09-21T14:24:07.626196+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:07.626196+0000","last_undegraded":"2024-09-21T14:24:07.626196+0000","last_fullsized":"2024-09-21T14:24:07.626196+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3911,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.248806+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:07.248806+0000","last_peered":"2024-09-21T14:24:07.248806+0000","last_clean":"2024-09-21T14:24:07.248806+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:07.248806+0000","last_undegraded":"2024-09-21T14:24:07.248806+0000","last_fullsized":"2024-09-21T14:24:07.248806+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:32.841741+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5158,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625092+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:07.625092+0000","last_peered":"2024-09-21T14:24:07.625092+0000","last_clean":"2024-09-21T14:24:07.625092+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:07.625092+0000","last_undegraded":"2024-09-21T14:24:07.625092+0000","last_fullsized":"2024-09-21T14:24:07.625092+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:05.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3588,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.572927+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:08.572927+0000","last_peered":"2024-09-21T14:24:08.572927+0000","last_clean":"2024-09-21T14:24:08.572927+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:08.572927+0000","last_undegraded":"2024-09-21T14:24:08.572927+0000","last_fullsized":"2024-09-21T14:24:08.572927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:11.998270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3361,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259197+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:08.259197+0000","last_peered":"2024-09-21T14:24:08.259197+0000","last_clean":"2024-09-21T14:24:08.259197+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:08.259197+0000","last_undegraded":"2024-09-21T14:24:08.259197+0000","last_fullsized":"2024-09-21T14:24:08.259197+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:21.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3775,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T14:24:07.625902+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:07.625902+0000","last_peered":"2024-09-21T14:24:07.625902+0000","last_clean":"2024-09-21T14:24:07.625902+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:07.625902+0000","last_undegraded":"2024-09-21T14:24:07.625902+0000","last_fullsized":"2024-09-21T14:24:07.625902+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4888,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T14:24:08.259038+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:08.259038+0000","last_peered":"2024-09-21T14:24:08.259038+0000","last_clean":"2024-09-21T14:24:08.259038+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:08.259038+0000","last_undegraded":"2024-09-21T14:24:08.259038+0000","last_fullsized":"2024-09-21T14:24:08.259038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:38.269031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.433,"15min":0.432},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.439},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.624,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.432,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.497}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.34200000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.625,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58199999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.52300000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.747,"15min":0.747},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2248704,"data_stored":1655571,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.432,"15min":0.432},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65500000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59999999999999998,"5min":2.593,"15min":2.593},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.441,"15min":0.45700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46300000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.72299999999999998,"15min":0.77400000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.47199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52400000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59499999999999997,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":2,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.61099999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64700000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44900000000000001,"15min":0.45500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:23:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59548,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312292,"statfs":{"total":96636764160,"available":96575787008,"internally_reserved":0,"allocated":2244608,"data_stored":1662438,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53200000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32400000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.47399999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.499},{"interface":"front","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.442}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71299999999999997,"5min":1.714,"15min":1.714},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.319,"15min":1.319},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29160,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342680,"statfs":{"total":96636764160,"available":96606904320,"internally_reserved":0,"allocated":2183168,"data_stored":1066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.34399999999999997,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.44800000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.38500000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46899999999999997,"5min":0.621,"15min":0.621},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.44900000000000001,"15min":0.46899999999999997},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.55800000000000005,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.442},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":2.097,"15min":2.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:23:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2195456,"data_stored":1071103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.217},"max":{"1min":0.495,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.251},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.24099999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32200000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.308}]},{"osd":4,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.35599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:23:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65252,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306588,"statfs":{"total":96636764160,"available":96569946112,"internally_reserved":0,"allocated":2187264,"data_stored":1066189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.432},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.216,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.34799999999999998,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39800000000000002,"15min":0.41199999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45800000000000002,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":1.292,"15min":1.292},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":1.341,"15min":1.341},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61699999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67900000000000005,"5min":1.284,"15min":1.284},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 14:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":1.034,"15min":1.034},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":1.254,"15min":1.254},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:14.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:24:14.385 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:14.385 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":183,"fsid":"da87574b-7df0-407c-8805-347c49d2d055","created":"2024-09-21T14:20:56.341644+0000","modified":"2024-09-21T14:24:11.273256+0000","last_up_change":"2024-09-21T14:21:06.519050+0000","last_in_change":"2024-09-21T14:23:48.408443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-21T14:21:07.255609+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-21T14:21:08.449794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:21:16.638770+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":97,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4cd103e2-3667-49ee-9ab4-f3b9e9a18d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6801","nonce":3182191515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6803","nonce":3182191515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6807","nonce":3182191515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3182191515},{"type":"v1","addr":"172.21.15.73:6805","nonce":3182191515}]},"public_addr":"172.21.15.73:6801/3182191515","cluster_addr":"172.21.15.73:6803/3182191515","heartbeat_back_addr":"172.21.15.73:6807/3182191515","heartbeat_front_addr":"172.21.15.73:6805/3182191515","state":["exists","up"]},{"osd":1,"uuid":"b7cc4ae1-1649-4592-91e7-4814ad9aa260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6809","nonce":3526445386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6811","nonce":3526445386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6815","nonce":3526445386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3526445386},{"type":"v1","addr":"172.21.15.73:6813","nonce":3526445386}]},"public_addr":"172.21.15.73:6809/3526445386","cluster_addr":"172.21.15.73:6811/3526445386","heartbeat_back_addr":"172.21.15.73:6815/3526445386","heartbeat_front_addr":"172.21.15.73:6813/3526445386","state":["exists","up"]},{"osd":2,"uuid":"adda4736-6878-49a4-ae7c-77b97c866703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6817","nonce":3383683073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6819","nonce":3383683073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6823","nonce":3383683073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3383683073},{"type":"v1","addr":"172.21.15.73:6821","nonce":3383683073}]},"public_addr":"172.21.15.73:6817/3383683073","cluster_addr":"172.21.15.73:6819/3383683073","heartbeat_back_addr":"172.21.15.73:6823/3383683073","heartbeat_front_addr":"172.21.15.73:6821/3383683073","state":["exists","up"]},{"osd":3,"uuid":"af765f77-37b5-4efd-89f0-4dfe5ef0978f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6825","nonce":3204248728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6827","nonce":3204248728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6831","nonce":3204248728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":3204248728},{"type":"v1","addr":"172.21.15.73:6829","nonce":3204248728}]},"public_addr":"172.21.15.73:6825/3204248728","cluster_addr":"172.21.15.73:6827/3204248728","heartbeat_back_addr":"172.21.15.73:6831/3204248728","heartbeat_front_addr":"172.21.15.73:6829/3204248728","state":["exists","up"]},{"osd":4,"uuid":"8a1c392f-d4d6-438c-91b6-a51fa7a6fa6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6801","nonce":3444590304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6803","nonce":3444590304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6807","nonce":3444590304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3444590304},{"type":"v1","addr":"172.21.15.161:6805","nonce":3444590304}]},"public_addr":"172.21.15.161:6801/3444590304","cluster_addr":"172.21.15.161:6803/3444590304","heartbeat_back_addr":"172.21.15.161:6807/3444590304","heartbeat_front_addr":"172.21.15.161:6805/3444590304","state":["exists","up"]},{"osd":5,"uuid":"5daf6d5d-18e7-475d-87ae-4b15965a8192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6809","nonce":4067730938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6811","nonce":4067730938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6815","nonce":4067730938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4067730938},{"type":"v1","addr":"172.21.15.161:6813","nonce":4067730938}]},"public_addr":"172.21.15.161:6809/4067730938","cluster_addr":"172.21.15.161:6811/4067730938","heartbeat_back_addr":"172.21.15.161:6815/4067730938","heartbeat_front_addr":"172.21.15.161:6813/4067730938","state":["exists","up"]},{"osd":6,"uuid":"e75e58a4-a37f-4029-b4fb-7ba22db2d6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6817","nonce":2060683966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6819","nonce":2060683966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6823","nonce":2060683966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2060683966},{"type":"v1","addr":"172.21.15.161:6821","nonce":2060683966}]},"public_addr":"172.21.15.161:6817/2060683966","cluster_addr":"172.21.15.161:6819/2060683966","heartbeat_back_addr":"172.21.15.161:6823/2060683966","heartbeat_front_addr":"172.21.15.161:6821/2060683966","state":["exists","up"]},{"osd":7,"uuid":"c0378ee4-c4f7-4430-ac1e-5740e2e8bfae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6825","nonce":2285860342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6827","nonce":2285860342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6831","nonce":2285860342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2285860342},{"type":"v1","addr":"172.21.15.161:6829","nonce":2285860342}]},"public_addr":"172.21.15.161:6825/2285860342","cluster_addr":"172.21.15.161:6827/2285860342","heartbeat_back_addr":"172.21.15.161:6831/2285860342","heartbeat_front_addr":"172.21.15.161:6829/2285860342","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.729613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.843490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.767547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.797830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.760985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.830324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.747245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:21:03.849223+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.1","osds":[5,4]},{"pgid":"3.3","osds":[6,5]},{"pgid":"3.d","osds":[6,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:24:15.398 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T14:24:15.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:15.527 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:15.527 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:15.527 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:15.538 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T14:24:15.809 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T14:24:15.822 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T14:24:15.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:15.943 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:15.943 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:15.943 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:15.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T14:24:16.225 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T14:24:16.237 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T14:24:16.237 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:16.359 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:16.359 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:16.359 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:16.369 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T14:24:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T14:24:16.655 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T14:24:16.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:16.775 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:16.775 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:16.775 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:16.786 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T14:24:17.055 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T14:24:17.068 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T14:24:17.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:17.189 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:17.189 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:17.190 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:17.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T14:24:17.473 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T14:24:17.486 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T14:24:17.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:17.607 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:17.608 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:17.608 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:17.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T14:24:17.889 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T14:24:17.902 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T14:24:17.902 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:18.023 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:18.023 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:18.023 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:18.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T14:24:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T14:24:18.321 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T14:24:18.321 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T14:24:18.442 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-21T14:24:18.442 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T14:24:18.442 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-21T14:24:18.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T14:24:18.723 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T14:24:18.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:19.007 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:19.007 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:19.018 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-21T14:24:18.448479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19566,"ondisk_log_size":19566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399180,"kb_used_data":17368,"kb_used_omap":12,"kb_used_meta":381683,"kb_avail":754575540,"statfs":{"total":773094113280,"available":772685352960,"internally_reserved":0,"allocated":17784832,"data_stored":9802600,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":390843961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.182167"},"pg_stats":[{"pgid":"3.1f","version":"159'396","reported_seq":3713,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.132149+0000","last_change":"2024-09-21T14:23:46.155237+0000","last_active":"2024-09-21T14:24:10.132149+0000","last_peered":"2024-09-21T14:24:10.132149+0000","last_clean":"2024-09-21T14:24:10.132149+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:10.132149+0000","last_undegraded":"2024-09-21T14:24:10.132149+0000","last_fullsized":"2024-09-21T14:24:10.132149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:20.118572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"159'343","reported_seq":2730,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.132107+0000","last_change":"2024-09-21T14:23:53.168116+0000","last_active":"2024-09-21T14:24:10.132107+0000","last_peered":"2024-09-21T14:24:10.132107+0000","last_clean":"2024-09-21T14:24:10.132107+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:10.132107+0000","last_undegraded":"2024-09-21T14:24:10.132107+0000","last_fullsized":"2024-09-21T14:24:10.132107+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:32.343687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"155'449","reported_seq":3560,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279561+0000","last_change":"2024-09-21T14:23:46.167953+0000","last_active":"2024-09-21T14:24:11.279561+0000","last_peered":"2024-09-21T14:24:11.279561+0000","last_clean":"2024-09-21T14:24:11.279561+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:11.279561+0000","last_undegraded":"2024-09-21T14:24:11.279561+0000","last_fullsized":"2024-09-21T14:24:11.279561+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"159'518","reported_seq":6358,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469671+0000","last_change":"2024-09-21T14:23:46.160573+0000","last_active":"2024-09-21T14:24:11.469671+0000","last_peered":"2024-09-21T14:24:11.469671+0000","last_clean":"2024-09-21T14:24:11.469671+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:11.469671+0000","last_undegraded":"2024-09-21T14:24:11.469671+0000","last_fullsized":"2024-09-21T14:24:11.469671+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"157'567","reported_seq":5015,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270145+0000","last_change":"2024-09-21T14:23:46.169462+0000","last_active":"2024-09-21T14:24:09.270145+0000","last_peered":"2024-09-21T14:24:09.270145+0000","last_clean":"2024-09-21T14:24:09.270145+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:09.270145+0000","last_undegraded":"2024-09-21T14:24:09.270145+0000","last_fullsized":"2024-09-21T14:24:09.270145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:09.108591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5851,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469723+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:11.469723+0000","last_peered":"2024-09-21T14:24:11.469723+0000","last_clean":"2024-09-21T14:24:11.469723+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:11.469723+0000","last_undegraded":"2024-09-21T14:24:11.469723+0000","last_fullsized":"2024-09-21T14:24:11.469723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"158'843","reported_seq":6267,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270092+0000","last_change":"2024-09-21T14:23:46.169777+0000","last_active":"2024-09-21T14:24:09.270092+0000","last_peered":"2024-09-21T14:24:09.270092+0000","last_clean":"2024-09-21T14:24:09.270092+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:09.270092+0000","last_undegraded":"2024-09-21T14:24:09.270092+0000","last_fullsized":"2024-09-21T14:24:09.270092+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:14.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'298","reported_seq":2160,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.132054+0000","last_change":"2024-09-21T14:23:46.154523+0000","last_active":"2024-09-21T14:24:10.132054+0000","last_peered":"2024-09-21T14:24:10.132054+0000","last_clean":"2024-09-21T14:24:10.132054+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:10.132054+0000","last_undegraded":"2024-09-21T14:24:10.132054+0000","last_fullsized":"2024-09-21T14:24:10.132054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:34.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"155'833","reported_seq":3738,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.132173+0000","last_change":"2024-09-21T14:23:46.155294+0000","last_active":"2024-09-21T14:24:10.132173+0000","last_peered":"2024-09-21T14:24:10.132173+0000","last_clean":"2024-09-21T14:24:10.132173+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:10.132173+0000","last_undegraded":"2024-09-21T14:24:10.132173+0000","last_fullsized":"2024-09-21T14:24:10.132173+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:08.333966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'533","reported_seq":2986,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.280150+0000","last_change":"2024-09-21T14:23:53.167479+0000","last_active":"2024-09-21T14:24:11.280150+0000","last_peered":"2024-09-21T14:24:11.280150+0000","last_clean":"2024-09-21T14:24:11.280150+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:11.280150+0000","last_undegraded":"2024-09-21T14:24:11.280150+0000","last_fullsized":"2024-09-21T14:24:11.280150+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"157'827","reported_seq":3097,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.373921+0000","last_change":"2024-09-21T14:23:56.177686+0000","last_active":"2024-09-21T14:24:11.373921+0000","last_peered":"2024-09-21T14:24:11.373921+0000","last_clean":"2024-09-21T14:24:11.373921+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:11.373921+0000","last_undegraded":"2024-09-21T14:24:11.373921+0000","last_fullsized":"2024-09-21T14:24:11.373921+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'516","reported_seq":3890,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279496+0000","last_change":"2024-09-21T14:23:53.167895+0000","last_active":"2024-09-21T14:24:11.279496+0000","last_peered":"2024-09-21T14:24:11.279496+0000","last_clean":"2024-09-21T14:24:11.279496+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:11.279496+0000","last_undegraded":"2024-09-21T14:24:11.279496+0000","last_fullsized":"2024-09-21T14:24:11.279496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469713+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:11.469713+0000","last_peered":"2024-09-21T14:24:11.469713+0000","last_clean":"2024-09-21T14:24:11.469713+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:11.469713+0000","last_undegraded":"2024-09-21T14:24:11.469713+0000","last_fullsized":"2024-09-21T14:24:11.469713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'476","reported_seq":3309,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.280148+0000","last_change":"2024-09-21T14:23:53.168932+0000","last_active":"2024-09-21T14:24:11.280148+0000","last_peered":"2024-09-21T14:24:11.280148+0000","last_clean":"2024-09-21T14:24:11.280148+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:11.280148+0000","last_undegraded":"2024-09-21T14:24:11.280148+0000","last_fullsized":"2024-09-21T14:24:11.280148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279635+0000","last_change":"2024-09-21T14:23:30.327789+0000","last_active":"2024-09-21T14:24:11.279635+0000","last_peered":"2024-09-21T14:24:11.279635+0000","last_clean":"2024-09-21T14:24:11.279635+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:11.279635+0000","last_undegraded":"2024-09-21T14:24:11.279635+0000","last_fullsized":"2024-09-21T14:24:11.279635+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'554","reported_seq":3448,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.356522+0000","last_change":"2024-09-21T14:23:46.158766+0000","last_active":"2024-09-21T14:24:10.356522+0000","last_peered":"2024-09-21T14:24:10.356522+0000","last_clean":"2024-09-21T14:24:10.356522+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:10.356522+0000","last_undegraded":"2024-09-21T14:24:10.356522+0000","last_fullsized":"2024-09-21T14:24:10.356522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:12.033086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:17.091779+0000","last_change":"2024-09-21T14:24:17.091882+0000","last_active":"2024-09-21T14:24:17.091779+0000","last_peered":"2024-09-21T14:24:17.091779+0000","last_clean":"2024-09-21T14:24:17.091779+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:17.091779+0000","last_undegraded":"2024-09-21T14:24:17.091779+0000","last_fullsized":"2024-09-21T14:24:17.091779+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_clean_scrub_stamp":"2024-09-21T14:24:17.091760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:04.663740+0000","scrub_duration":0.001950645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"155'409","reported_seq":3323,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270135+0000","last_change":"2024-09-21T14:23:46.169516+0000","last_active":"2024-09-21T14:24:09.270135+0000","last_peered":"2024-09-21T14:24:09.270135+0000","last_clean":"2024-09-21T14:24:09.270135+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:09.270135+0000","last_undegraded":"2024-09-21T14:24:09.270135+0000","last_fullsized":"2024-09-21T14:24:09.270135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:25:05.611544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":176,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.374787+0000","last_change":"2024-09-21T14:24:11.374787+0000","last_active":"2024-09-21T14:24:11.374787+0000","last_peered":"2024-09-21T14:24:11.374787+0000","last_clean":"2024-09-21T14:24:11.374787+0000","last_became_active":"2024-09-21T14:24:11.374541+0000","last_became_peered":"2024-09-21T14:24:11.374541+0000","last_unstale":"2024-09-21T14:24:11.374787+0000","last_undegraded":"2024-09-21T14:24:11.374787+0000","last_fullsized":"2024-09-21T14:24:11.374787+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:09.485025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"159'590","reported_seq":5777,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270090+0000","last_change":"2024-09-21T14:23:46.169924+0000","last_active":"2024-09-21T14:24:09.270090+0000","last_peered":"2024-09-21T14:24:09.270090+0000","last_clean":"2024-09-21T14:24:09.270090+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:09.270090+0000","last_undegraded":"2024-09-21T14:24:09.270090+0000","last_fullsized":"2024-09-21T14:24:09.270090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:23.677729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"155'385","reported_seq":3263,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.356433+0000","last_change":"2024-09-21T14:23:52.163772+0000","last_active":"2024-09-21T14:24:10.356433+0000","last_peered":"2024-09-21T14:24:10.356433+0000","last_clean":"2024-09-21T14:24:10.356433+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:10.356433+0000","last_undegraded":"2024-09-21T14:24:10.356433+0000","last_fullsized":"2024-09-21T14:24:10.356433+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:43.132161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":293,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279378+0000","last_change":"2024-09-21T14:23:33.164038+0000","last_active":"2024-09-21T14:24:11.279378+0000","last_peered":"2024-09-21T14:24:11.279378+0000","last_clean":"2024-09-21T14:24:11.279378+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:11.279378+0000","last_undegraded":"2024-09-21T14:24:11.279378+0000","last_fullsized":"2024-09-21T14:24:11.279378+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:08.522820+0000","last_clean_scrub_stamp":"2024-09-21T14:21:08.522820+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":207,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270127+0000","last_change":"2024-09-21T14:23:36.166146+0000","last_active":"2024-09-21T14:24:09.270127+0000","last_peered":"2024-09-21T14:24:09.270127+0000","last_clean":"2024-09-21T14:24:09.270127+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:09.270127+0000","last_undegraded":"2024-09-21T14:24:09.270127+0000","last_fullsized":"2024-09-21T14:24:09.270127+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:01.194281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"157'797","reported_seq":3527,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.132024+0000","last_change":"2024-09-21T14:23:46.154520+0000","last_active":"2024-09-21T14:24:10.132024+0000","last_peered":"2024-09-21T14:24:10.132024+0000","last_clean":"2024-09-21T14:24:10.132024+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:10.132024+0000","last_undegraded":"2024-09-21T14:24:10.132024+0000","last_fullsized":"2024-09-21T14:24:10.132024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:30.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":161,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.373841+0000","last_change":"2024-09-21T14:23:56.176847+0000","last_active":"2024-09-21T14:24:11.373841+0000","last_peered":"2024-09-21T14:24:11.373841+0000","last_clean":"2024-09-21T14:24:11.373841+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:11.373841+0000","last_undegraded":"2024-09-21T14:24:11.373841+0000","last_fullsized":"2024-09-21T14:24:11.373841+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"159'1256","reported_seq":3008,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.356454+0000","last_change":"2024-09-21T14:23:46.159164+0000","last_active":"2024-09-21T14:24:10.356454+0000","last_peered":"2024-09-21T14:24:10.356454+0000","last_clean":"2024-09-21T14:24:10.356454+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:10.356454+0000","last_undegraded":"2024-09-21T14:24:10.356454+0000","last_fullsized":"2024-09-21T14:24:10.356454+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:37.525134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":128,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.280296+0000","last_change":"2024-09-21T14:22:08.453127+0000","last_active":"2024-09-21T14:24:11.280296+0000","last_peered":"2024-09-21T14:24:11.280296+0000","last_clean":"2024-09-21T14:24:11.280296+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:11.280296+0000","last_undegraded":"2024-09-21T14:24:11.280296+0000","last_fullsized":"2024-09-21T14:24:11.280296+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:15.335516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'609","reported_seq":3054,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.270036+0000","last_change":"2024-09-21T14:23:53.165937+0000","last_active":"2024-09-21T14:24:09.270036+0000","last_peered":"2024-09-21T14:24:09.270036+0000","last_clean":"2024-09-21T14:24:09.270036+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:09.270036+0000","last_undegraded":"2024-09-21T14:24:09.270036+0000","last_fullsized":"2024-09-21T14:24:09.270036+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:48.717568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469691+0000","last_change":"2024-09-21T14:23:36.159151+0000","last_active":"2024-09-21T14:24:11.469691+0000","last_peered":"2024-09-21T14:24:11.469691+0000","last_clean":"2024-09-21T14:24:11.469691+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:11.469691+0000","last_undegraded":"2024-09-21T14:24:11.469691+0000","last_fullsized":"2024-09-21T14:24:11.469691+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'685","reported_seq":3832,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:09.269987+0000","last_change":"2024-09-21T14:23:46.169105+0000","last_active":"2024-09-21T14:24:09.269987+0000","last_peered":"2024-09-21T14:24:09.269987+0000","last_clean":"2024-09-21T14:24:09.269987+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:09.269987+0000","last_undegraded":"2024-09-21T14:24:09.269987+0000","last_fullsized":"2024-09-21T14:24:09.269987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:34.417553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"159'703","reported_seq":4811,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279480+0000","last_change":"2024-09-21T14:23:46.167452+0000","last_active":"2024-09-21T14:24:11.279480+0000","last_peered":"2024-09-21T14:24:11.279480+0000","last_clean":"2024-09-21T14:24:11.279480+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:11.279480+0000","last_undegraded":"2024-09-21T14:24:11.279480+0000","last_fullsized":"2024-09-21T14:24:11.279480+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"158'481","reported_seq":2363,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279539+0000","last_change":"2024-09-21T14:23:46.167009+0000","last_active":"2024-09-21T14:24:11.279539+0000","last_peered":"2024-09-21T14:24:11.279539+0000","last_clean":"2024-09-21T14:24:11.279539+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:11.279539+0000","last_undegraded":"2024-09-21T14:24:11.279539+0000","last_fullsized":"2024-09-21T14:24:11.279539+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'481","reported_seq":2562,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.289701+0000","last_change":"2024-09-21T14:24:11.287874+0000","last_active":"2024-09-21T14:24:11.289701+0000","last_peered":"2024-09-21T14:24:11.289701+0000","last_clean":"2024-09-21T14:24:11.289701+0000","last_became_active":"2024-09-21T14:24:11.286380+0000","last_became_peered":"2024-09-21T14:24:11.286380+0000","last_unstale":"2024-09-21T14:24:11.289701+0000","last_undegraded":"2024-09-21T14:24:11.289701+0000","last_fullsized":"2024-09-21T14:24:11.289701+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.776836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044423299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"159'743","reported_seq":2621,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.373856+0000","last_change":"2024-09-21T14:23:46.163682+0000","last_active":"2024-09-21T14:24:11.373856+0000","last_peered":"2024-09-21T14:24:11.373856+0000","last_clean":"2024-09-21T14:24:11.373856+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:11.373856+0000","last_undegraded":"2024-09-21T14:24:11.373856+0000","last_fullsized":"2024-09-21T14:24:11.373856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"157'570","reported_seq":3096,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.356524+0000","last_change":"2024-09-21T14:23:53.170794+0000","last_active":"2024-09-21T14:24:10.356524+0000","last_peered":"2024-09-21T14:24:10.356524+0000","last_clean":"2024-09-21T14:24:10.356524+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:10.356524+0000","last_undegraded":"2024-09-21T14:24:10.356524+0000","last_fullsized":"2024-09-21T14:24:10.356524+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:09.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"156'998","reported_seq":3914,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.280195+0000","last_change":"2024-09-21T14:23:53.170282+0000","last_active":"2024-09-21T14:24:11.280195+0000","last_peered":"2024-09-21T14:24:11.280195+0000","last_clean":"2024-09-21T14:24:11.280195+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:11.280195+0000","last_undegraded":"2024-09-21T14:24:11.280195+0000","last_fullsized":"2024-09-21T14:24:11.280195+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"159'517","reported_seq":5161,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.373942+0000","last_change":"2024-09-21T14:23:56.178269+0000","last_active":"2024-09-21T14:24:11.373942+0000","last_peered":"2024-09-21T14:24:11.373942+0000","last_clean":"2024-09-21T14:24:11.373942+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:11.373942+0000","last_undegraded":"2024-09-21T14:24:11.373942+0000","last_fullsized":"2024-09-21T14:24:11.373942+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3590,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469556+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:11.469556+0000","last_peered":"2024-09-21T14:24:11.469556+0000","last_clean":"2024-09-21T14:24:11.469556+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:11.469556+0000","last_undegraded":"2024-09-21T14:24:11.469556+0000","last_fullsized":"2024-09-21T14:24:11.469556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"159'383","reported_seq":3363,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279382+0000","last_change":"2024-09-21T14:23:52.164363+0000","last_active":"2024-09-21T14:24:11.279382+0000","last_peered":"2024-09-21T14:24:11.279382+0000","last_clean":"2024-09-21T14:24:11.279382+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:11.279382+0000","last_undegraded":"2024-09-21T14:24:11.279382+0000","last_fullsized":"2024-09-21T14:24:11.279382+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"155'549","reported_seq":3777,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-21T14:24:10.356390+0000","last_change":"2024-09-21T14:23:46.158328+0000","last_active":"2024-09-21T14:24:10.356390+0000","last_peered":"2024-09-21T14:24:10.356390+0000","last_clean":"2024-09-21T14:24:10.356390+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:10.356390+0000","last_undegraded":"2024-09-21T14:24:10.356390+0000","last_fullsized":"2024-09-21T14:24:10.356390+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:09.656915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4890,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279462+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:11.279462+0000","last_peered":"2024-09-21T14:24:11.279462+0000","last_clean":"2024-09-21T14:24:11.279462+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:11.279462+0000","last_undegraded":"2024-09-21T14:24:11.279462+0000","last_fullsized":"2024-09-21T14:24:11.279462+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19446,"ondisk_log_size":19446,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55420,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316420,"statfs":{"total":96636764160,"available":96580014080,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.434},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.61899999999999999,"5min":0.749,"15min":0.749},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.67700000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46700000000000003,"15min":0.45800000000000002},"min":{"1min":0.33000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.435},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.441,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.627,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.68100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81200000000000006,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39000000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.373,"15min":0.37},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.61699999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42699999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.503}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.42299999999999999},"min":{"1min":0.32700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66700000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45900000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.35099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.621,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.41599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46000000000000002},"min":{"1min":0.36499999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.498},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.32500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.27800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.57599999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.624},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0189999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59899999999999998,"15min":0.65500000000000003},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63500000000000001,"5min":0.747,"15min":0.747},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44600000000000001,"15min":0.45100000000000001},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66800000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60799999999999998,"5min":0.747,"15min":0.747},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.47699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59576,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312264,"statfs":{"total":96636764160,"available":96575758336,"internally_reserved":0,"allocated":2273280,"data_stored":1667371,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.439}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.316,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72599999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68700000000000006,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.495}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42599999999999999,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47799999999999998,"15min":0.47099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63800000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":2.593,"15min":2.593},"last":0.42699999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.86199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.66000000000000003,"15min":0.753},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.46500000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.48799999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41499999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59516,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312324,"statfs":{"total":96636764160,"available":96575819776,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.436},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.73799999999999999,"5min":0.746,"15min":0.746},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.27600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.71499999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.437,"15min":0.443},"min":{"1min":0.26400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.499},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.0109999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.42799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59564,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312276,"statfs":{"total":96636764160,"available":96575770624,"internally_reserved":0,"allocated":2256896,"data_stored":1668338,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.312,"15min":0.311},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.34699999999999998},"min":{"1min":0.25700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46400000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.28599999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.21099999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.372,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.23000000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.498,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.30199999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.626},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69599999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79500000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67400000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.54800000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.752,"5min":1.714,"15min":1.714},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76800000000000002,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":1.319,"15min":1.319},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29180,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342660,"statfs":{"total":96636764160,"available":96606883840,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.20799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.42899999999999999,"15min":0.45900000000000002},"min":{"1min":0.24099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.466,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.35499999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.315}]},{"osd":3,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.436,"15min":0.46400000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.6100000000000001,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.429,"5min":2.097,"15min":2.097},"last":0.373}]},{"osd":4,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.754},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.55600000000000005}]},{"osd":7,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2207744,"data_stored":1077003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.311,"15min":0.30599999999999999},"min":{"1min":0.20499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.31},{"interface":"front","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.21199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.34699999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.36199999999999999,"15min":0.35799999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38400000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.49199999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.374}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.44900000000000001},"min":{"1min":0.33200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51100000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.373,"15min":0.373},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65276,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306564,"statfs":{"total":96636764160,"available":96569921536,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.34000000000000002,"15min":0.35399999999999998},"min":{"1min":0.17899999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.45400000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36299999999999999,"15min":0.37},"min":{"1min":0.28100000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.55800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":1.236}]},{"osd":2,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33400000000000002,"15min":0.34799999999999998},"min":{"1min":0.245,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.28}]},{"osd":3,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38200000000000001,"15min":0.40600000000000003},"min":{"1min":0.214,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53500000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":1.131},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.443,"15min":0.46600000000000003},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.1519999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55300000000000005,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59399999999999997,"5min":1.292,"15min":1.292},"last":1.016},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.59399999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":1.341,"15min":1.341},"last":1.214}]},{"osd":5,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.36099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":1.284,"15min":1.284},"last":0.745},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.258}]},{"osd":7,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74299999999999999,"5min":1.254,"15min":1.254},"last":0.98099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:19.023 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.023 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.023 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.023 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.024 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T14:21:08.522820+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.025 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.026 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:19.027 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T14:24:39.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:39.314 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:39.315 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:39.326 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-21T14:24:38.452889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19920,"ondisk_log_size":19920,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":403224,"kb_used_data":17368,"kb_used_omap":12,"kb_used_meta":385779,"kb_avail":754571496,"statfs":{"total":773094113280,"available":772681211904,"internally_reserved":0,"allocated":17784832,"data_stored":9802600,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":395038265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002687"},"pg_stats":[{"pgid":"3.1f","version":"183'408","reported_seq":3737,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:32.581609+0000","last_change":"2024-09-21T14:24:32.581720+0000","last_active":"2024-09-21T14:24:32.581609+0000","last_peered":"2024-09-21T14:24:32.581609+0000","last_clean":"2024-09-21T14:24:32.581609+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:32.581609+0000","last_undegraded":"2024-09-21T14:24:32.581609+0000","last_fullsized":"2024-09-21T14:24:32.581609+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'408","last_scrub_stamp":"2024-09-21T14:24:32.581586+0000","last_deep_scrub":"183'408","last_deep_scrub_stamp":"2024-09-21T14:24:32.581586+0000","last_clean_scrub_stamp":"2024-09-21T14:24:32.581586+0000","objects_scrubbed":12,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:57:52.355791+0000","scrub_duration":0.0092072660000000004,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"183'354","reported_seq":2753,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:20.527790+0000","last_change":"2024-09-21T14:24:20.527931+0000","last_active":"2024-09-21T14:24:20.527790+0000","last_peered":"2024-09-21T14:24:20.527790+0000","last_clean":"2024-09-21T14:24:20.527790+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:20.527790+0000","last_undegraded":"2024-09-21T14:24:20.527790+0000","last_fullsized":"2024-09-21T14:24:20.527790+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'354","last_scrub_stamp":"2024-09-21T14:24:20.527766+0000","last_deep_scrub":"183'354","last_deep_scrub_stamp":"2024-09-21T14:24:20.527766+0000","last_clean_scrub_stamp":"2024-09-21T14:24:20.527766+0000","objects_scrubbed":11,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:37:19.835883+0000","scrub_duration":0.0089071059999999997,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"183'462","reported_seq":3584,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:31.809229+0000","last_change":"2024-09-21T14:24:31.809354+0000","last_active":"2024-09-21T14:24:31.809229+0000","last_peered":"2024-09-21T14:24:31.809229+0000","last_clean":"2024-09-21T14:24:31.809229+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:31.809229+0000","last_undegraded":"2024-09-21T14:24:31.809229+0000","last_fullsized":"2024-09-21T14:24:31.809229+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'462","last_scrub_stamp":"2024-09-21T14:24:31.809206+0000","last_deep_scrub":"183'462","last_deep_scrub_stamp":"2024-09-21T14:24:31.809206+0000","last_clean_scrub_stamp":"2024-09-21T14:24:31.809206+0000","objects_scrubbed":13,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:51.500751+0000","scrub_duration":0.016182384000000001,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"183'530","reported_seq":6381,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:30.585801+0000","last_change":"2024-09-21T14:24:30.585960+0000","last_active":"2024-09-21T14:24:30.585801+0000","last_peered":"2024-09-21T14:24:30.585801+0000","last_clean":"2024-09-21T14:24:30.585801+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:30.585801+0000","last_undegraded":"2024-09-21T14:24:30.585801+0000","last_fullsized":"2024-09-21T14:24:30.585801+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'530","last_scrub_stamp":"2024-09-21T14:24:30.585776+0000","last_deep_scrub":"183'530","last_deep_scrub_stamp":"2024-09-21T14:24:30.585776+0000","last_clean_scrub_stamp":"2024-09-21T14:24:30.585776+0000","objects_scrubbed":12,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:06:17.978525+0000","scrub_duration":0.0094062539999999993,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"183'579","reported_seq":5039,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.125132+0000","last_change":"2024-09-21T14:24:28.125263+0000","last_active":"2024-09-21T14:24:28.125132+0000","last_peered":"2024-09-21T14:24:28.125132+0000","last_clean":"2024-09-21T14:24:28.125132+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:28.125132+0000","last_undegraded":"2024-09-21T14:24:28.125132+0000","last_fullsized":"2024-09-21T14:24:28.125132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'579","last_scrub_stamp":"2024-09-21T14:24:28.125109+0000","last_deep_scrub":"183'579","last_deep_scrub_stamp":"2024-09-21T14:24:28.125109+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.125109+0000","objects_scrubbed":12,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:32.458466+0000","scrub_duration":0.014464157,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"157'572","reported_seq":5851,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469723+0000","last_change":"2024-09-21T14:23:46.160964+0000","last_active":"2024-09-21T14:24:11.469723+0000","last_peered":"2024-09-21T14:24:11.469723+0000","last_clean":"2024-09-21T14:24:11.469723+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:11.469723+0000","last_undegraded":"2024-09-21T14:24:11.469723+0000","last_fullsized":"2024-09-21T14:24:11.469723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"183'852","reported_seq":6288,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.137814+0000","last_change":"2024-09-21T14:24:24.137946+0000","last_active":"2024-09-21T14:24:24.137814+0000","last_peered":"2024-09-21T14:24:24.137814+0000","last_clean":"2024-09-21T14:24:24.137814+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:24.137814+0000","last_undegraded":"2024-09-21T14:24:24.137814+0000","last_fullsized":"2024-09-21T14:24:24.137814+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'852","last_scrub_stamp":"2024-09-21T14:24:24.137788+0000","last_deep_scrub":"183'852","last_deep_scrub_stamp":"2024-09-21T14:24:24.137788+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.137788+0000","objects_scrubbed":9,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:43.532811+0000","scrub_duration":0.010248626,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"183'308","reported_seq":2182,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:18.589728+0000","last_change":"2024-09-21T14:24:18.589859+0000","last_active":"2024-09-21T14:24:18.589728+0000","last_peered":"2024-09-21T14:24:18.589728+0000","last_clean":"2024-09-21T14:24:18.589728+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:18.589728+0000","last_undegraded":"2024-09-21T14:24:18.589728+0000","last_fullsized":"2024-09-21T14:24:18.589728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'308","last_scrub_stamp":"2024-09-21T14:24:18.589704+0000","last_deep_scrub":"183'308","last_deep_scrub_stamp":"2024-09-21T14:24:18.589704+0000","last_clean_scrub_stamp":"2024-09-21T14:24:18.589704+0000","objects_scrubbed":10,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:25.639377+0000","scrub_duration":0.0083773259999999992,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"183'843","reported_seq":3760,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:23.512786+0000","last_change":"2024-09-21T14:24:23.512921+0000","last_active":"2024-09-21T14:24:23.512786+0000","last_peered":"2024-09-21T14:24:23.512786+0000","last_clean":"2024-09-21T14:24:23.512786+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:23.512786+0000","last_undegraded":"2024-09-21T14:24:23.512786+0000","last_fullsized":"2024-09-21T14:24:23.512786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'843","last_scrub_stamp":"2024-09-21T14:24:23.512760+0000","last_deep_scrub":"183'843","last_deep_scrub_stamp":"2024-09-21T14:24:23.512760+0000","last_clean_scrub_stamp":"2024-09-21T14:24:23.512760+0000","objects_scrubbed":10,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:59.251112+0000","scrub_duration":0.011703135,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"183'546","reported_seq":3010,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.125511+0000","last_change":"2024-09-21T14:24:29.125644+0000","last_active":"2024-09-21T14:24:29.125511+0000","last_peered":"2024-09-21T14:24:29.125511+0000","last_clean":"2024-09-21T14:24:29.125511+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:29.125511+0000","last_undegraded":"2024-09-21T14:24:29.125511+0000","last_fullsized":"2024-09-21T14:24:29.125511+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'546","last_scrub_stamp":"2024-09-21T14:24:29.125489+0000","last_deep_scrub":"183'546","last_deep_scrub_stamp":"2024-09-21T14:24:29.125489+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.125489+0000","objects_scrubbed":13,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:08.567320+0000","scrub_duration":0.014286665,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"183'842","reported_seq":3123,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.966556+0000","last_change":"2024-09-21T14:24:25.966715+0000","last_active":"2024-09-21T14:24:25.966556+0000","last_peered":"2024-09-21T14:24:25.966556+0000","last_clean":"2024-09-21T14:24:25.966556+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:25.966556+0000","last_undegraded":"2024-09-21T14:24:25.966556+0000","last_fullsized":"2024-09-21T14:24:25.966556+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'842","last_scrub_stamp":"2024-09-21T14:24:25.966533+0000","last_deep_scrub":"183'842","last_deep_scrub_stamp":"2024-09-21T14:24:25.966533+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.966533+0000","objects_scrubbed":15,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:01.716678+0000","scrub_duration":0.011731271,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"183'535","reported_seq":3920,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.673193+0000","last_change":"2024-09-21T14:24:21.673410+0000","last_active":"2024-09-21T14:24:21.673193+0000","last_peered":"2024-09-21T14:24:21.673193+0000","last_clean":"2024-09-21T14:24:21.673193+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:21.673193+0000","last_undegraded":"2024-09-21T14:24:21.673193+0000","last_fullsized":"2024-09-21T14:24:21.673193+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'535","last_scrub_stamp":"2024-09-21T14:24:21.673170+0000","last_deep_scrub":"183'535","last_deep_scrub_stamp":"2024-09-21T14:24:21.673170+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.673170+0000","objects_scrubbed":19,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:46.414800+0000","scrub_duration":0.016381968,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469713+0000","last_change":"2024-09-21T14:23:36.156547+0000","last_active":"2024-09-21T14:24:11.469713+0000","last_peered":"2024-09-21T14:24:11.469713+0000","last_clean":"2024-09-21T14:24:11.469713+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:11.469713+0000","last_undegraded":"2024-09-21T14:24:11.469713+0000","last_fullsized":"2024-09-21T14:24:11.469713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:07.520787+0000","last_clean_scrub_stamp":"2024-09-21T14:21:07.520787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"183'492","reported_seq":3336,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:18.070734+0000","last_change":"2024-09-21T14:24:18.070840+0000","last_active":"2024-09-21T14:24:18.070734+0000","last_peered":"2024-09-21T14:24:18.070734+0000","last_clean":"2024-09-21T14:24:18.070734+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:18.070734+0000","last_undegraded":"2024-09-21T14:24:18.070734+0000","last_fullsized":"2024-09-21T14:24:18.070734+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'492","last_scrub_stamp":"2024-09-21T14:24:18.070715+0000","last_deep_scrub":"183'492","last_deep_scrub_stamp":"2024-09-21T14:24:18.070715+0000","last_clean_scrub_stamp":"2024-09-21T14:24:18.070715+0000","objects_scrubbed":16,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:14.362228+0000","scrub_duration":0.010502641,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.710105+0000","last_change":"2024-09-21T14:24:19.710199+0000","last_active":"2024-09-21T14:24:19.710105+0000","last_peered":"2024-09-21T14:24:19.710105+0000","last_clean":"2024-09-21T14:24:19.710105+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:19.710105+0000","last_undegraded":"2024-09-21T14:24:19.710105+0000","last_fullsized":"2024-09-21T14:24:19.710105+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:19.710087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:19.710087+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.710087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:33:10.419406+0000","scrub_duration":0.0014355399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"183'565","reported_seq":3471,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.911564+0000","last_change":"2024-09-21T14:24:26.911676+0000","last_active":"2024-09-21T14:24:26.911564+0000","last_peered":"2024-09-21T14:24:26.911564+0000","last_clean":"2024-09-21T14:24:26.911564+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:26.911564+0000","last_undegraded":"2024-09-21T14:24:26.911564+0000","last_fullsized":"2024-09-21T14:24:26.911564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'565","last_scrub_stamp":"2024-09-21T14:24:26.911541+0000","last_deep_scrub":"183'565","last_deep_scrub_stamp":"2024-09-21T14:24:26.911541+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.911541+0000","objects_scrubbed":11,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:02.368271+0000","scrub_duration":0.0097215419999999997,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:17.091779+0000","last_change":"2024-09-21T14:24:17.091882+0000","last_active":"2024-09-21T14:24:17.091779+0000","last_peered":"2024-09-21T14:24:17.091779+0000","last_clean":"2024-09-21T14:24:17.091779+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:17.091779+0000","last_undegraded":"2024-09-21T14:24:17.091779+0000","last_fullsized":"2024-09-21T14:24:17.091779+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_clean_scrub_stamp":"2024-09-21T14:24:17.091760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:04.663740+0000","scrub_duration":0.001950645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"183'422","reported_seq":3348,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.106223+0000","last_change":"2024-09-21T14:24:21.106344+0000","last_active":"2024-09-21T14:24:21.106223+0000","last_peered":"2024-09-21T14:24:21.106223+0000","last_clean":"2024-09-21T14:24:21.106223+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:21.106223+0000","last_undegraded":"2024-09-21T14:24:21.106223+0000","last_fullsized":"2024-09-21T14:24:21.106223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'422","last_scrub_stamp":"2024-09-21T14:24:21.106201+0000","last_deep_scrub":"183'422","last_deep_scrub_stamp":"2024-09-21T14:24:21.106201+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.106201+0000","objects_scrubbed":13,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:08.289431+0000","scrub_duration":0.013243446000000001,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":188,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:22.667114+0000","last_change":"2024-09-21T14:24:22.667223+0000","last_active":"2024-09-21T14:24:22.667114+0000","last_peered":"2024-09-21T14:24:22.667114+0000","last_clean":"2024-09-21T14:24:22.667114+0000","last_became_active":"2024-09-21T14:24:11.374541+0000","last_became_peered":"2024-09-21T14:24:11.374541+0000","last_unstale":"2024-09-21T14:24:22.667114+0000","last_undegraded":"2024-09-21T14:24:22.667114+0000","last_fullsized":"2024-09-21T14:24:22.667114+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:22.667093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:22.667093+0000","last_clean_scrub_stamp":"2024-09-21T14:24:22.667093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:02.215249+0000","scrub_duration":0.0019808209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"183'602","reported_seq":5801,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.094867+0000","last_change":"2024-09-21T14:24:26.095027+0000","last_active":"2024-09-21T14:24:26.094867+0000","last_peered":"2024-09-21T14:24:26.094867+0000","last_clean":"2024-09-21T14:24:26.094867+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:26.094867+0000","last_undegraded":"2024-09-21T14:24:26.094867+0000","last_fullsized":"2024-09-21T14:24:26.094867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'602","last_scrub_stamp":"2024-09-21T14:24:26.094840+0000","last_deep_scrub":"183'602","last_deep_scrub_stamp":"2024-09-21T14:24:26.094840+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.094840+0000","objects_scrubbed":12,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:38:52.820394+0000","scrub_duration":0.013029986,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"183'392","reported_seq":3282,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:20.957002+0000","last_change":"2024-09-21T14:24:20.957131+0000","last_active":"2024-09-21T14:24:20.957002+0000","last_peered":"2024-09-21T14:24:20.957002+0000","last_clean":"2024-09-21T14:24:20.957002+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:20.957002+0000","last_undegraded":"2024-09-21T14:24:20.957002+0000","last_fullsized":"2024-09-21T14:24:20.957002+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'392","last_scrub_stamp":"2024-09-21T14:24:20.956979+0000","last_deep_scrub":"183'392","last_deep_scrub_stamp":"2024-09-21T14:24:20.956979+0000","last_clean_scrub_stamp":"2024-09-21T14:24:20.956979+0000","objects_scrubbed":7,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:30.917718+0000","scrub_duration":0.010057023999999999,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"183'119","reported_seq":306,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:22.689468+0000","last_change":"2024-09-21T14:24:22.689578+0000","last_active":"2024-09-21T14:24:22.689468+0000","last_peered":"2024-09-21T14:24:22.689468+0000","last_clean":"2024-09-21T14:24:22.689468+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:22.689468+0000","last_undegraded":"2024-09-21T14:24:22.689468+0000","last_fullsized":"2024-09-21T14:24:22.689468+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'119","last_scrub_stamp":"2024-09-21T14:24:22.689444+0000","last_deep_scrub":"183'119","last_deep_scrub_stamp":"2024-09-21T14:24:22.689444+0000","last_clean_scrub_stamp":"2024-09-21T14:24:22.689444+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:53.339918+0000","scrub_duration":0.0072623640000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"183'2","reported_seq":220,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.061272+0000","last_change":"2024-09-21T14:24:19.061392+0000","last_active":"2024-09-21T14:24:19.061272+0000","last_peered":"2024-09-21T14:24:19.061272+0000","last_clean":"2024-09-21T14:24:19.061272+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:19.061272+0000","last_undegraded":"2024-09-21T14:24:19.061272+0000","last_fullsized":"2024-09-21T14:24:19.061272+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'2","last_scrub_stamp":"2024-09-21T14:24:19.061250+0000","last_deep_scrub":"183'2","last_deep_scrub_stamp":"2024-09-21T14:24:19.061250+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.061250+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:42.514797+0000","scrub_duration":0.0036601849999999998,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"183'816","reported_seq":3558,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.514131+0000","last_change":"2024-09-21T14:24:25.514243+0000","last_active":"2024-09-21T14:24:25.514131+0000","last_peered":"2024-09-21T14:24:25.514131+0000","last_clean":"2024-09-21T14:24:25.514131+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:25.514131+0000","last_undegraded":"2024-09-21T14:24:25.514131+0000","last_fullsized":"2024-09-21T14:24:25.514131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'816","last_scrub_stamp":"2024-09-21T14:24:25.514108+0000","last_deep_scrub":"183'816","last_deep_scrub_stamp":"2024-09-21T14:24:25.514108+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.514108+0000","objects_scrubbed":19,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:29.643670+0000","scrub_duration":0.017727863,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"183'3","reported_seq":173,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.928154+0000","last_change":"2024-09-21T14:24:24.928283+0000","last_active":"2024-09-21T14:24:24.928154+0000","last_peered":"2024-09-21T14:24:24.928154+0000","last_clean":"2024-09-21T14:24:24.928154+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:24.928154+0000","last_undegraded":"2024-09-21T14:24:24.928154+0000","last_fullsized":"2024-09-21T14:24:24.928154+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'3","last_scrub_stamp":"2024-09-21T14:24:24.928131+0000","last_deep_scrub":"183'3","last_deep_scrub_stamp":"2024-09-21T14:24:24.928131+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.928131+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:49:14.281753+0000","scrub_duration":0.0038982660000000001,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"183'1264","reported_seq":3028,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.908677+0000","last_change":"2024-09-21T14:24:28.908797+0000","last_active":"2024-09-21T14:24:28.908677+0000","last_peered":"2024-09-21T14:24:28.908677+0000","last_clean":"2024-09-21T14:24:28.908677+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:28.908677+0000","last_undegraded":"2024-09-21T14:24:28.908677+0000","last_fullsized":"2024-09-21T14:24:28.908677+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'1264","last_scrub_stamp":"2024-09-21T14:24:28.908652+0000","last_deep_scrub":"183'1264","last_deep_scrub_stamp":"2024-09-21T14:24:28.908652+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.908652+0000","objects_scrubbed":8,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:33:16.414971+0000","scrub_duration":0.0086359590000000003,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":140,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:33.555277+0000","last_change":"2024-09-21T14:24:33.555392+0000","last_active":"2024-09-21T14:24:33.555277+0000","last_peered":"2024-09-21T14:24:33.555277+0000","last_clean":"2024-09-21T14:24:33.555277+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:33.555277+0000","last_undegraded":"2024-09-21T14:24:33.555277+0000","last_fullsized":"2024-09-21T14:24:33.555277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:33.555255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:33.555255+0000","last_clean_scrub_stamp":"2024-09-21T14:24:33.555255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:36.660361+0000","scrub_duration":0.001445265,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"183'623","reported_seq":3080,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:23.121148+0000","last_change":"2024-09-21T14:24:23.121272+0000","last_active":"2024-09-21T14:24:23.121148+0000","last_peered":"2024-09-21T14:24:23.121148+0000","last_clean":"2024-09-21T14:24:23.121148+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:23.121148+0000","last_undegraded":"2024-09-21T14:24:23.121148+0000","last_fullsized":"2024-09-21T14:24:23.121148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'623","last_scrub_stamp":"2024-09-21T14:24:23.121124+0000","last_deep_scrub":"183'623","last_deep_scrub_stamp":"2024-09-21T14:24:23.121124+0000","last_clean_scrub_stamp":"2024-09-21T14:24:23.121124+0000","objects_scrubbed":14,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:02.221894+0000","scrub_duration":0.011825186,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.571302+0000","last_change":"2024-09-21T14:24:28.571442+0000","last_active":"2024-09-21T14:24:28.571302+0000","last_peered":"2024-09-21T14:24:28.571302+0000","last_clean":"2024-09-21T14:24:28.571302+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:28.571302+0000","last_undegraded":"2024-09-21T14:24:28.571302+0000","last_fullsized":"2024-09-21T14:24:28.571302+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:28.571280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:28.571280+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.571280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:14.893355+0000","scrub_duration":0.0019501270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"183'692","reported_seq":3851,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.165956+0000","last_change":"2024-09-21T14:24:29.166130+0000","last_active":"2024-09-21T14:24:29.165956+0000","last_peered":"2024-09-21T14:24:29.165956+0000","last_clean":"2024-09-21T14:24:29.165956+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:29.165956+0000","last_undegraded":"2024-09-21T14:24:29.165956+0000","last_fullsized":"2024-09-21T14:24:29.165956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'692","last_scrub_stamp":"2024-09-21T14:24:29.165932+0000","last_deep_scrub":"183'692","last_deep_scrub_stamp":"2024-09-21T14:24:29.165932+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.165932+0000","objects_scrubbed":7,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:34.075288+0000","scrub_duration":0.0079997139999999998,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"183'714","reported_seq":4833,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:27.649445+0000","last_change":"2024-09-21T14:24:27.649552+0000","last_active":"2024-09-21T14:24:27.649445+0000","last_peered":"2024-09-21T14:24:27.649445+0000","last_clean":"2024-09-21T14:24:27.649445+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:27.649445+0000","last_undegraded":"2024-09-21T14:24:27.649445+0000","last_fullsized":"2024-09-21T14:24:27.649445+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'714","last_scrub_stamp":"2024-09-21T14:24:27.649422+0000","last_deep_scrub":"183'714","last_deep_scrub_stamp":"2024-09-21T14:24:27.649422+0000","last_clean_scrub_stamp":"2024-09-21T14:24:27.649422+0000","objects_scrubbed":11,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:37.569259+0000","scrub_duration":0.0090103430000000005,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"183'493","reported_seq":2386,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:33.791893+0000","last_change":"2024-09-21T14:24:33.792022+0000","last_active":"2024-09-21T14:24:33.791893+0000","last_peered":"2024-09-21T14:24:33.791893+0000","last_clean":"2024-09-21T14:24:33.791893+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:33.791893+0000","last_undegraded":"2024-09-21T14:24:33.791893+0000","last_fullsized":"2024-09-21T14:24:33.791893+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'493","last_scrub_stamp":"2024-09-21T14:24:33.791867+0000","last_deep_scrub":"183'493","last_deep_scrub_stamp":"2024-09-21T14:24:33.791867+0000","last_clean_scrub_stamp":"2024-09-21T14:24:33.791867+0000","objects_scrubbed":12,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:23:56.649520+0000","scrub_duration":0.0099566480000000002,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"183'491","reported_seq":2584,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.713952+0000","last_change":"2024-09-21T14:24:19.714091+0000","last_active":"2024-09-21T14:24:19.713952+0000","last_peered":"2024-09-21T14:24:19.713952+0000","last_clean":"2024-09-21T14:24:19.713952+0000","last_became_active":"2024-09-21T14:24:11.286380+0000","last_became_peered":"2024-09-21T14:24:11.286380+0000","last_unstale":"2024-09-21T14:24:19.713952+0000","last_undegraded":"2024-09-21T14:24:19.713952+0000","last_fullsized":"2024-09-21T14:24:19.713952+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'491","last_scrub_stamp":"2024-09-21T14:24:19.713929+0000","last_deep_scrub":"183'491","last_deep_scrub_stamp":"2024-09-21T14:24:19.713929+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.713929+0000","objects_scrubbed":10,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:08.648885+0000","scrub_duration":0.0091389910000000008,"objects_trimmed":0,"snaptrim_duration":0.00044423299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"183'755","reported_seq":2644,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.875096+0000","last_change":"2024-09-21T14:24:21.875206+0000","last_active":"2024-09-21T14:24:21.875096+0000","last_peered":"2024-09-21T14:24:21.875096+0000","last_clean":"2024-09-21T14:24:21.875096+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:21.875096+0000","last_undegraded":"2024-09-21T14:24:21.875096+0000","last_fullsized":"2024-09-21T14:24:21.875096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'755","last_scrub_stamp":"2024-09-21T14:24:21.875072+0000","last_deep_scrub":"183'755","last_deep_scrub_stamp":"2024-09-21T14:24:21.875072+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.875072+0000","objects_scrubbed":12,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:49.964907+0000","scrub_duration":0.014754738999999999,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"183'578","reported_seq":3116,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.953850+0000","last_change":"2024-09-21T14:24:29.953981+0000","last_active":"2024-09-21T14:24:29.953850+0000","last_peered":"2024-09-21T14:24:29.953850+0000","last_clean":"2024-09-21T14:24:29.953850+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:29.953850+0000","last_undegraded":"2024-09-21T14:24:29.953850+0000","last_fullsized":"2024-09-21T14:24:29.953850+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'578","last_scrub_stamp":"2024-09-21T14:24:29.953817+0000","last_deep_scrub":"183'578","last_deep_scrub_stamp":"2024-09-21T14:24:29.953817+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.953817+0000","objects_scrubbed":8,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:04:53.938190+0000","scrub_duration":0.0068265540000000003,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"183'1009","reported_seq":3936,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.137471+0000","last_change":"2024-09-21T14:24:24.137641+0000","last_active":"2024-09-21T14:24:24.137471+0000","last_peered":"2024-09-21T14:24:24.137471+0000","last_clean":"2024-09-21T14:24:24.137471+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:24.137471+0000","last_undegraded":"2024-09-21T14:24:24.137471+0000","last_fullsized":"2024-09-21T14:24:24.137471+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'1009","last_scrub_stamp":"2024-09-21T14:24:24.137430+0000","last_deep_scrub":"183'1009","last_deep_scrub_stamp":"2024-09-21T14:24:24.137430+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.137430+0000","objects_scrubbed":11,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:01:18.332693+0000","scrub_duration":0.011649343,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"183'530","reported_seq":5185,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:27.981932+0000","last_change":"2024-09-21T14:24:27.982058+0000","last_active":"2024-09-21T14:24:27.981932+0000","last_peered":"2024-09-21T14:24:27.981932+0000","last_clean":"2024-09-21T14:24:27.981932+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:27.981932+0000","last_undegraded":"2024-09-21T14:24:27.981932+0000","last_fullsized":"2024-09-21T14:24:27.981932+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'530","last_scrub_stamp":"2024-09-21T14:24:27.981907+0000","last_deep_scrub":"183'530","last_deep_scrub_stamp":"2024-09-21T14:24:27.981907+0000","last_clean_scrub_stamp":"2024-09-21T14:24:27.981907+0000","objects_scrubbed":13,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:05.795907+0000","scrub_duration":0.015791745999999999,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"157'440","reported_seq":3590,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.469556+0000","last_change":"2024-09-21T14:23:46.160129+0000","last_active":"2024-09-21T14:24:11.469556+0000","last_peered":"2024-09-21T14:24:11.469556+0000","last_clean":"2024-09-21T14:24:11.469556+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:11.469556+0000","last_undegraded":"2024-09-21T14:24:11.469556+0000","last_fullsized":"2024-09-21T14:24:11.469556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'401","reported_seq":3392,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.668381+0000","last_change":"2024-09-21T14:24:26.668505+0000","last_active":"2024-09-21T14:24:26.668381+0000","last_peered":"2024-09-21T14:24:26.668381+0000","last_clean":"2024-09-21T14:24:26.668381+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:26.668381+0000","last_undegraded":"2024-09-21T14:24:26.668381+0000","last_fullsized":"2024-09-21T14:24:26.668381+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'401","last_scrub_stamp":"2024-09-21T14:24:26.668357+0000","last_deep_scrub":"183'401","last_deep_scrub_stamp":"2024-09-21T14:24:26.668357+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.668357+0000","objects_scrubbed":18,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:09:57.562717+0000","scrub_duration":0.012675017,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"183'561","reported_seq":3801,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.928031+0000","last_change":"2024-09-21T14:24:25.928166+0000","last_active":"2024-09-21T14:24:25.928031+0000","last_peered":"2024-09-21T14:24:25.928031+0000","last_clean":"2024-09-21T14:24:25.928031+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:25.928031+0000","last_undegraded":"2024-09-21T14:24:25.928031+0000","last_fullsized":"2024-09-21T14:24:25.928031+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'561","last_scrub_stamp":"2024-09-21T14:24:25.928006+0000","last_deep_scrub":"183'561","last_deep_scrub_stamp":"2024-09-21T14:24:25.928006+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.928006+0000","objects_scrubbed":12,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:18.286938+0000","scrub_duration":0.0097054910000000001,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"157'1125","reported_seq":4890,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:11.279462+0000","last_change":"2024-09-21T14:23:46.168059+0000","last_active":"2024-09-21T14:24:11.279462+0000","last_peered":"2024-09-21T14:24:11.279462+0000","last_clean":"2024-09-21T14:24:11.279462+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:11.279462+0000","last_undegraded":"2024-09-21T14:24:11.279462+0000","last_fullsized":"2024-09-21T14:24:11.279462+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:21:17.126174+0000","last_clean_scrub_stamp":"2024-09-21T14:21:17.126174+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19796,"ondisk_log_size":19796,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55412,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316428,"statfs":{"total":96636764160,"available":96580022272,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.434},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.61899999999999999,"5min":0.749,"15min":0.749},"last":0.435},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.67700000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53100000000000003}]},{"osd":1,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46700000000000003,"15min":0.45800000000000002},"min":{"1min":0.33000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.435},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.441,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.627,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.68100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81200000000000006,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.35599999999999998}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.373,"15min":0.37},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.61699999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42699999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.375},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.42299999999999999},"min":{"1min":0.32700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66700000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.35099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.621,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.41599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.503}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46000000000000002},"min":{"1min":0.36499999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.32500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58699999999999997}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.27800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.57599999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0189999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59899999999999998,"15min":0.65500000000000003},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.372},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.32100000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63500000000000001,"5min":0.747,"15min":0.747},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44600000000000001,"15min":0.45100000000000001},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66800000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47799999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60799999999999998,"5min":0.747,"15min":0.747},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.47699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.495}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59572,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312268,"statfs":{"total":96636764160,"available":96575762432,"internally_reserved":0,"allocated":2273280,"data_stored":1667371,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.44900000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.316,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72599999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68700000000000006,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67800000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42599999999999999,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.435}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47799999999999998,"15min":0.47099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63800000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":2.593,"15min":2.593},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.86199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.66000000000000003,"15min":0.753},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.622}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.48799999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69999999999999996}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59512,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.436},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.73799999999999999,"5min":0.746,"15min":0.746},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.27600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47799999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.71499999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42599999999999999}]},{"osd":3,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.437,"15min":0.443},"min":{"1min":0.26400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.373}]},{"osd":6,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.502}]},{"osd":7,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.499},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.0109999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.41799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2256896,"data_stored":1668338,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.312,"15min":0.311},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.437},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.34699999999999998},"min":{"1min":0.25700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46400000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46000000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.21099999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.372,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.23000000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39300000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.498,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.503}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69599999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79500000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67400000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.752,"5min":1.714,"15min":1.714},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76800000000000002,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":1.319,"15min":1.319},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29172,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342668,"statfs":{"total":96636764160,"available":96606892032,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.20799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.433,"5min":0.42899999999999999,"15min":0.45900000000000002},"min":{"1min":0.24099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.466,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.30599999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.36699999999999999}]},{"osd":3,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.436,"15min":0.46400000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.6100000000000001,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.429,"5min":2.097,"15min":2.097},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.57299999999999995}]},{"osd":6,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2207744,"data_stored":1077003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.311,"15min":0.30599999999999999},"min":{"1min":0.20499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.432},{"interface":"front","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.21199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.36199999999999999,"15min":0.35799999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53100000000000003}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.49199999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51200000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.44900000000000001},"min":{"1min":0.33200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.90400000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.373,"15min":0.373},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.93200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65272,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306568,"statfs":{"total":96636764160,"available":96569925632,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.34000000000000002,"15min":0.35399999999999998},"min":{"1min":0.17899999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.45400000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36299999999999999,"15min":0.37},"min":{"1min":0.28100000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.55800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33400000000000002,"15min":0.34799999999999998},"min":{"1min":0.245,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.436},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47399999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38200000000000001,"15min":0.40600000000000003},"min":{"1min":0.214,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53500000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.443,"15min":0.46600000000000003},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.503}]},{"osd":4,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55300000000000005,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59399999999999997,"5min":1.292,"15min":1.292},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.59399999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":1.341,"15min":1.341},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.36099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":1.284,"15min":1.284},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74299999999999999,"5min":1.254,"15min":1.254},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:39.329 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:39.329 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-21T14:21:07.520787+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=7, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:39.329 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:39.330 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T14:21:17.126174+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=21, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=24, tm_sec=14, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T14:24:39.330 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T14:24:59.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:24:59.619 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-21T14:24:59.619 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-21T14:24:59.629 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-21T14:24:58.457410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":380,"num_read":149137,"num_read_kb":954939,"num_write":22020,"num_write_kb":506181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6237,"num_bytes_recovered":283966457,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19954,"ondisk_log_size":19954,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":403224,"kb_used_data":17368,"kb_used_omap":12,"kb_used_meta":385779,"kb_avail":754571496,"statfs":{"total":773094113280,"available":772681211904,"internally_reserved":0,"allocated":17784832,"data_stored":9802600,"data_compressed":66455,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":395038265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002611"},"pg_stats":[{"pgid":"3.1f","version":"183'408","reported_seq":3737,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:32.581609+0000","last_change":"2024-09-21T14:24:32.581720+0000","last_active":"2024-09-21T14:24:32.581609+0000","last_peered":"2024-09-21T14:24:32.581609+0000","last_clean":"2024-09-21T14:24:32.581609+0000","last_became_active":"2024-09-21T14:23:39.373465+0000","last_became_peered":"2024-09-21T14:23:39.373465+0000","last_unstale":"2024-09-21T14:24:32.581609+0000","last_undegraded":"2024-09-21T14:24:32.581609+0000","last_fullsized":"2024-09-21T14:24:32.581609+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'408","last_scrub_stamp":"2024-09-21T14:24:32.581586+0000","last_deep_scrub":"183'408","last_deep_scrub_stamp":"2024-09-21T14:24:32.581586+0000","last_clean_scrub_stamp":"2024-09-21T14:24:32.581586+0000","objects_scrubbed":12,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:57:52.355791+0000","scrub_duration":0.0092072660000000004,"objects_trimmed":0,"snaptrim_duration":0.001395791,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3823,"num_read_kb":17766,"num_write":360,"num_write_kb":4795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":3880066,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"183'354","reported_seq":2753,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:20.527790+0000","last_change":"2024-09-21T14:24:20.527931+0000","last_active":"2024-09-21T14:24:20.527790+0000","last_peered":"2024-09-21T14:24:20.527790+0000","last_clean":"2024-09-21T14:24:20.527790+0000","last_became_active":"2024-09-21T14:23:53.167224+0000","last_became_peered":"2024-09-21T14:23:53.167224+0000","last_unstale":"2024-09-21T14:24:20.527790+0000","last_undegraded":"2024-09-21T14:24:20.527790+0000","last_fullsized":"2024-09-21T14:24:20.527790+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'354","last_scrub_stamp":"2024-09-21T14:24:20.527766+0000","last_deep_scrub":"183'354","last_deep_scrub_stamp":"2024-09-21T14:24:20.527766+0000","last_clean_scrub_stamp":"2024-09-21T14:24:20.527766+0000","objects_scrubbed":11,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:37:19.835883+0000","scrub_duration":0.0089071059999999997,"objects_trimmed":0,"snaptrim_duration":0.001276253,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2119,"num_read_kb":53411,"num_write":322,"num_write_kb":35431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":32397316,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"183'462","reported_seq":3584,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:31.809229+0000","last_change":"2024-09-21T14:24:31.809354+0000","last_active":"2024-09-21T14:24:31.809229+0000","last_peered":"2024-09-21T14:24:31.809229+0000","last_clean":"2024-09-21T14:24:31.809229+0000","last_became_active":"2024-09-21T14:23:34.159266+0000","last_became_peered":"2024-09-21T14:23:34.159266+0000","last_unstale":"2024-09-21T14:24:31.809229+0000","last_undegraded":"2024-09-21T14:24:31.809229+0000","last_fullsized":"2024-09-21T14:24:31.809229+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'462","last_scrub_stamp":"2024-09-21T14:24:31.809206+0000","last_deep_scrub":"183'462","last_deep_scrub_stamp":"2024-09-21T14:24:31.809206+0000","last_clean_scrub_stamp":"2024-09-21T14:24:31.809206+0000","objects_scrubbed":13,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:12:51.500751+0000","scrub_duration":0.016182384000000001,"objects_trimmed":0,"snaptrim_duration":0.0018181810000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3947,"num_read_kb":20095,"num_write":439,"num_write_kb":10363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":14823912,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"183'530","reported_seq":6381,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:30.585801+0000","last_change":"2024-09-21T14:24:30.585960+0000","last_active":"2024-09-21T14:24:30.585801+0000","last_peered":"2024-09-21T14:24:30.585801+0000","last_clean":"2024-09-21T14:24:30.585801+0000","last_became_active":"2024-09-21T14:23:39.368282+0000","last_became_peered":"2024-09-21T14:23:39.368282+0000","last_unstale":"2024-09-21T14:24:30.585801+0000","last_undegraded":"2024-09-21T14:24:30.585801+0000","last_fullsized":"2024-09-21T14:24:30.585801+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'530","last_scrub_stamp":"2024-09-21T14:24:30.585776+0000","last_deep_scrub":"183'530","last_deep_scrub_stamp":"2024-09-21T14:24:30.585776+0000","last_clean_scrub_stamp":"2024-09-21T14:24:30.585776+0000","objects_scrubbed":12,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:06:17.978525+0000","scrub_duration":0.0094062539999999993,"objects_trimmed":0,"snaptrim_duration":0.00095751899999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6548,"num_read_kb":83163,"num_write":559,"num_write_kb":30705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":28728215,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"183'579","reported_seq":5039,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.125132+0000","last_change":"2024-09-21T14:24:28.125263+0000","last_active":"2024-09-21T14:24:28.125132+0000","last_peered":"2024-09-21T14:24:28.125132+0000","last_clean":"2024-09-21T14:24:28.125132+0000","last_became_active":"2024-09-21T14:21:18.148885+0000","last_became_peered":"2024-09-21T14:21:18.148885+0000","last_unstale":"2024-09-21T14:24:28.125132+0000","last_undegraded":"2024-09-21T14:24:28.125132+0000","last_fullsized":"2024-09-21T14:24:28.125132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'579","last_scrub_stamp":"2024-09-21T14:24:28.125109+0000","last_deep_scrub":"183'579","last_deep_scrub_stamp":"2024-09-21T14:24:28.125109+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.125109+0000","objects_scrubbed":12,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:32.458466+0000","scrub_duration":0.014464157,"objects_trimmed":0,"snaptrim_duration":0.00097451199999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6448,"num_read_kb":42904,"num_write":463,"num_write_kb":19682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"183'583","reported_seq":5873,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:38.504640+0000","last_change":"2024-09-21T14:24:38.504755+0000","last_active":"2024-09-21T14:24:38.504640+0000","last_peered":"2024-09-21T14:24:38.504640+0000","last_clean":"2024-09-21T14:24:38.504640+0000","last_became_active":"2024-09-21T14:23:40.367954+0000","last_became_peered":"2024-09-21T14:23:40.367954+0000","last_unstale":"2024-09-21T14:24:38.504640+0000","last_undegraded":"2024-09-21T14:24:38.504640+0000","last_fullsized":"2024-09-21T14:24:38.504640+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'583","last_scrub_stamp":"2024-09-21T14:24:38.504617+0000","last_deep_scrub":"183'583","last_deep_scrub_stamp":"2024-09-21T14:24:38.504617+0000","last_clean_scrub_stamp":"2024-09-21T14:24:38.504617+0000","objects_scrubbed":11,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:58.737802+0000","scrub_duration":0.0091561890000000003,"objects_trimmed":0,"snaptrim_duration":0.0012052,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7227,"num_read_kb":66230,"num_write":587,"num_write_kb":34397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":29370832,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"183'852","reported_seq":6288,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.137814+0000","last_change":"2024-09-21T14:24:24.137946+0000","last_active":"2024-09-21T14:24:24.137814+0000","last_peered":"2024-09-21T14:24:24.137814+0000","last_clean":"2024-09-21T14:24:24.137814+0000","last_became_active":"2024-09-21T14:23:39.371804+0000","last_became_peered":"2024-09-21T14:23:39.371804+0000","last_unstale":"2024-09-21T14:24:24.137814+0000","last_undegraded":"2024-09-21T14:24:24.137814+0000","last_fullsized":"2024-09-21T14:24:24.137814+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'852","last_scrub_stamp":"2024-09-21T14:24:24.137788+0000","last_deep_scrub":"183'852","last_deep_scrub_stamp":"2024-09-21T14:24:24.137788+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.137788+0000","objects_scrubbed":9,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:43.532811+0000","scrub_duration":0.010248626,"objects_trimmed":0,"snaptrim_duration":0.000938466,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7620,"num_read_kb":30420,"num_write":1089,"num_write_kb":11034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":7528917,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"183'308","reported_seq":2182,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:18.589728+0000","last_change":"2024-09-21T14:24:18.589859+0000","last_active":"2024-09-21T14:24:18.589728+0000","last_peered":"2024-09-21T14:24:18.589728+0000","last_clean":"2024-09-21T14:24:18.589728+0000","last_became_active":"2024-09-21T14:21:18.150403+0000","last_became_peered":"2024-09-21T14:21:18.150403+0000","last_unstale":"2024-09-21T14:24:18.589728+0000","last_undegraded":"2024-09-21T14:24:18.589728+0000","last_fullsized":"2024-09-21T14:24:18.589728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'308","last_scrub_stamp":"2024-09-21T14:24:18.589704+0000","last_deep_scrub":"183'308","last_deep_scrub_stamp":"2024-09-21T14:24:18.589704+0000","last_clean_scrub_stamp":"2024-09-21T14:24:18.589704+0000","objects_scrubbed":10,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:25.639377+0000","scrub_duration":0.0083773259999999992,"objects_trimmed":0,"snaptrim_duration":0.00089423400000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1667,"num_read_kb":12542,"num_write":245,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"183'843","reported_seq":3760,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:23.512786+0000","last_change":"2024-09-21T14:24:23.512921+0000","last_active":"2024-09-21T14:24:23.512786+0000","last_peered":"2024-09-21T14:24:23.512786+0000","last_clean":"2024-09-21T14:24:23.512786+0000","last_became_active":"2024-09-21T14:23:35.167743+0000","last_became_peered":"2024-09-21T14:23:35.167743+0000","last_unstale":"2024-09-21T14:24:23.512786+0000","last_undegraded":"2024-09-21T14:24:23.512786+0000","last_fullsized":"2024-09-21T14:24:23.512786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'843","last_scrub_stamp":"2024-09-21T14:24:23.512760+0000","last_deep_scrub":"183'843","last_deep_scrub_stamp":"2024-09-21T14:24:23.512760+0000","last_clean_scrub_stamp":"2024-09-21T14:24:23.512760+0000","objects_scrubbed":10,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:59.251112+0000","scrub_duration":0.011703135,"objects_trimmed":0,"snaptrim_duration":0.0013549250000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5066,"num_read_kb":10483,"num_write":1004,"num_write_kb":14003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":5315222,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"183'546","reported_seq":3010,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.125511+0000","last_change":"2024-09-21T14:24:29.125644+0000","last_active":"2024-09-21T14:24:29.125511+0000","last_peered":"2024-09-21T14:24:29.125511+0000","last_clean":"2024-09-21T14:24:29.125511+0000","last_became_active":"2024-09-21T14:23:53.165720+0000","last_became_peered":"2024-09-21T14:23:53.165720+0000","last_unstale":"2024-09-21T14:24:29.125511+0000","last_undegraded":"2024-09-21T14:24:29.125511+0000","last_fullsized":"2024-09-21T14:24:29.125511+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'546","last_scrub_stamp":"2024-09-21T14:24:29.125489+0000","last_deep_scrub":"183'546","last_deep_scrub_stamp":"2024-09-21T14:24:29.125489+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.125489+0000","objects_scrubbed":13,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:08.567320+0000","scrub_duration":0.014286665,"objects_trimmed":0,"snaptrim_duration":0.0014788379999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3439,"num_read_kb":14065,"num_write":660,"num_write_kb":7262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":3633325,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"183'842","reported_seq":3123,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.966556+0000","last_change":"2024-09-21T14:24:25.966715+0000","last_active":"2024-09-21T14:24:25.966556+0000","last_peered":"2024-09-21T14:24:25.966556+0000","last_clean":"2024-09-21T14:24:25.966556+0000","last_became_active":"2024-09-21T14:23:56.176169+0000","last_became_peered":"2024-09-21T14:23:56.176169+0000","last_unstale":"2024-09-21T14:24:25.966556+0000","last_undegraded":"2024-09-21T14:24:25.966556+0000","last_fullsized":"2024-09-21T14:24:25.966556+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'842","last_scrub_stamp":"2024-09-21T14:24:25.966533+0000","last_deep_scrub":"183'842","last_deep_scrub_stamp":"2024-09-21T14:24:25.966533+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.966533+0000","objects_scrubbed":15,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:01.716678+0000","scrub_duration":0.011731271,"objects_trimmed":2,"snaptrim_duration":0.0029555050000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5129,"num_read_kb":24865,"num_write":1088,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"183'535","reported_seq":3920,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.673193+0000","last_change":"2024-09-21T14:24:21.673410+0000","last_active":"2024-09-21T14:24:21.673193+0000","last_peered":"2024-09-21T14:24:21.673193+0000","last_clean":"2024-09-21T14:24:21.673193+0000","last_became_active":"2024-09-21T14:23:53.165861+0000","last_became_peered":"2024-09-21T14:23:53.165861+0000","last_unstale":"2024-09-21T14:24:21.673193+0000","last_undegraded":"2024-09-21T14:24:21.673193+0000","last_fullsized":"2024-09-21T14:24:21.673193+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'535","last_scrub_stamp":"2024-09-21T14:24:21.673170+0000","last_deep_scrub":"183'535","last_deep_scrub_stamp":"2024-09-21T14:24:21.673170+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.673170+0000","objects_scrubbed":19,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:46.414800+0000","scrub_duration":0.016381968,"objects_trimmed":0,"snaptrim_duration":0.00058159800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5568,"num_read_kb":17366,"num_write":523,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":7557160,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:41.509467+0000","last_change":"2024-09-21T14:24:41.509650+0000","last_active":"2024-09-21T14:24:41.509467+0000","last_peered":"2024-09-21T14:24:41.509467+0000","last_clean":"2024-09-21T14:24:41.509467+0000","last_became_active":"2024-09-21T14:23:36.156285+0000","last_became_peered":"2024-09-21T14:23:36.156285+0000","last_unstale":"2024-09-21T14:24:41.509467+0000","last_undegraded":"2024-09-21T14:24:41.509467+0000","last_fullsized":"2024-09-21T14:24:41.509467+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:41.509435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:41.509435+0000","last_clean_scrub_stamp":"2024-09-21T14:24:41.509435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:32.924073+0000","scrub_duration":0.0021548510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"183'492","reported_seq":3336,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:18.070734+0000","last_change":"2024-09-21T14:24:18.070840+0000","last_active":"2024-09-21T14:24:18.070734+0000","last_peered":"2024-09-21T14:24:18.070734+0000","last_clean":"2024-09-21T14:24:18.070734+0000","last_became_active":"2024-09-21T14:23:53.166432+0000","last_became_peered":"2024-09-21T14:23:53.166432+0000","last_unstale":"2024-09-21T14:24:18.070734+0000","last_undegraded":"2024-09-21T14:24:18.070734+0000","last_fullsized":"2024-09-21T14:24:18.070734+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'492","last_scrub_stamp":"2024-09-21T14:24:18.070715+0000","last_deep_scrub":"183'492","last_deep_scrub_stamp":"2024-09-21T14:24:18.070715+0000","last_clean_scrub_stamp":"2024-09-21T14:24:18.070715+0000","objects_scrubbed":16,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:14.362228+0000","scrub_duration":0.010502641,"objects_trimmed":0,"snaptrim_duration":0.0014310219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3756,"num_read_kb":13073,"num_write":427,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":575426,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.710105+0000","last_change":"2024-09-21T14:24:19.710199+0000","last_active":"2024-09-21T14:24:19.710105+0000","last_peered":"2024-09-21T14:24:19.710105+0000","last_clean":"2024-09-21T14:24:19.710105+0000","last_became_active":"2024-09-21T14:23:30.327509+0000","last_became_peered":"2024-09-21T14:23:30.327509+0000","last_unstale":"2024-09-21T14:24:19.710105+0000","last_undegraded":"2024-09-21T14:24:19.710105+0000","last_fullsized":"2024-09-21T14:24:19.710105+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:19.710087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:19.710087+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.710087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:33:10.419406+0000","scrub_duration":0.0014355399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"183'565","reported_seq":3471,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.911564+0000","last_change":"2024-09-21T14:24:26.911676+0000","last_active":"2024-09-21T14:24:26.911564+0000","last_peered":"2024-09-21T14:24:26.911564+0000","last_clean":"2024-09-21T14:24:26.911564+0000","last_became_active":"2024-09-21T14:21:18.146813+0000","last_became_peered":"2024-09-21T14:21:18.146813+0000","last_unstale":"2024-09-21T14:24:26.911564+0000","last_undegraded":"2024-09-21T14:24:26.911564+0000","last_fullsized":"2024-09-21T14:24:26.911564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'565","last_scrub_stamp":"2024-09-21T14:24:26.911541+0000","last_deep_scrub":"183'565","last_deep_scrub_stamp":"2024-09-21T14:24:26.911541+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.911541+0000","objects_scrubbed":11,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:02.368271+0000","scrub_duration":0.0097215419999999997,"objects_trimmed":0,"snaptrim_duration":0.00099807600000000008,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4244,"num_read_kb":18346,"num_write":644,"num_write_kb":5971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:17.091779+0000","last_change":"2024-09-21T14:24:17.091882+0000","last_active":"2024-09-21T14:24:17.091779+0000","last_peered":"2024-09-21T14:24:17.091779+0000","last_clean":"2024-09-21T14:24:17.091779+0000","last_became_active":"2024-09-21T14:23:49.430768+0000","last_became_peered":"2024-09-21T14:23:49.430768+0000","last_unstale":"2024-09-21T14:24:17.091779+0000","last_undegraded":"2024-09-21T14:24:17.091779+0000","last_fullsized":"2024-09-21T14:24:17.091779+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:17.091760+0000","last_clean_scrub_stamp":"2024-09-21T14:24:17.091760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:04.663740+0000","scrub_duration":0.001950645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"183'422","reported_seq":3348,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.106223+0000","last_change":"2024-09-21T14:24:21.106344+0000","last_active":"2024-09-21T14:24:21.106223+0000","last_peered":"2024-09-21T14:24:21.106223+0000","last_clean":"2024-09-21T14:24:21.106223+0000","last_became_active":"2024-09-21T14:21:18.146058+0000","last_became_peered":"2024-09-21T14:21:18.146058+0000","last_unstale":"2024-09-21T14:24:21.106223+0000","last_undegraded":"2024-09-21T14:24:21.106223+0000","last_fullsized":"2024-09-21T14:24:21.106223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'422","last_scrub_stamp":"2024-09-21T14:24:21.106201+0000","last_deep_scrub":"183'422","last_deep_scrub_stamp":"2024-09-21T14:24:21.106201+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.106201+0000","objects_scrubbed":13,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:08.289431+0000","scrub_duration":0.013243446000000001,"objects_trimmed":0,"snaptrim_duration":0.00080719899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2840,"num_read_kb":36192,"num_write":356,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":188,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:22.667114+0000","last_change":"2024-09-21T14:24:22.667223+0000","last_active":"2024-09-21T14:24:22.667114+0000","last_peered":"2024-09-21T14:24:22.667114+0000","last_clean":"2024-09-21T14:24:22.667114+0000","last_became_active":"2024-09-21T14:24:11.374541+0000","last_became_peered":"2024-09-21T14:24:11.374541+0000","last_unstale":"2024-09-21T14:24:22.667114+0000","last_undegraded":"2024-09-21T14:24:22.667114+0000","last_fullsized":"2024-09-21T14:24:22.667114+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:22.667093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:22.667093+0000","last_clean_scrub_stamp":"2024-09-21T14:24:22.667093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:02.215249+0000","scrub_duration":0.0019808209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"183'602","reported_seq":5801,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.094867+0000","last_change":"2024-09-21T14:24:26.095027+0000","last_active":"2024-09-21T14:24:26.094867+0000","last_peered":"2024-09-21T14:24:26.094867+0000","last_clean":"2024-09-21T14:24:26.094867+0000","last_became_active":"2024-09-21T14:21:18.145302+0000","last_became_peered":"2024-09-21T14:21:18.145302+0000","last_unstale":"2024-09-21T14:24:26.094867+0000","last_undegraded":"2024-09-21T14:24:26.094867+0000","last_fullsized":"2024-09-21T14:24:26.094867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'602","last_scrub_stamp":"2024-09-21T14:24:26.094840+0000","last_deep_scrub":"183'602","last_deep_scrub_stamp":"2024-09-21T14:24:26.094840+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.094840+0000","objects_scrubbed":12,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:38:52.820394+0000","scrub_duration":0.013029986,"objects_trimmed":0,"snaptrim_duration":0.00096737799999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6836,"num_read_kb":31517,"num_write":530,"num_write_kb":9260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"183'392","reported_seq":3282,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:20.957002+0000","last_change":"2024-09-21T14:24:20.957131+0000","last_active":"2024-09-21T14:24:20.957002+0000","last_peered":"2024-09-21T14:24:20.957002+0000","last_clean":"2024-09-21T14:24:20.957002+0000","last_became_active":"2024-09-21T14:23:52.162900+0000","last_became_peered":"2024-09-21T14:23:52.162900+0000","last_unstale":"2024-09-21T14:24:20.957002+0000","last_undegraded":"2024-09-21T14:24:20.957002+0000","last_fullsized":"2024-09-21T14:24:20.957002+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'392","last_scrub_stamp":"2024-09-21T14:24:20.956979+0000","last_deep_scrub":"183'392","last_deep_scrub_stamp":"2024-09-21T14:24:20.956979+0000","last_clean_scrub_stamp":"2024-09-21T14:24:20.956979+0000","objects_scrubbed":7,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:30.917718+0000","scrub_duration":0.010057023999999999,"objects_trimmed":0,"snaptrim_duration":0.00063891299999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3628,"num_read_kb":22038,"num_write":319,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":3692668,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"183'119","reported_seq":306,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:22.689468+0000","last_change":"2024-09-21T14:24:22.689578+0000","last_active":"2024-09-21T14:24:22.689468+0000","last_peered":"2024-09-21T14:24:22.689468+0000","last_clean":"2024-09-21T14:24:22.689468+0000","last_became_active":"2024-09-21T14:23:33.163636+0000","last_became_peered":"2024-09-21T14:23:33.163636+0000","last_unstale":"2024-09-21T14:24:22.689468+0000","last_undegraded":"2024-09-21T14:24:22.689468+0000","last_fullsized":"2024-09-21T14:24:22.689468+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'119","last_scrub_stamp":"2024-09-21T14:24:22.689444+0000","last_deep_scrub":"183'119","last_deep_scrub_stamp":"2024-09-21T14:24:22.689444+0000","last_clean_scrub_stamp":"2024-09-21T14:24:22.689444+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:53.339918+0000","scrub_duration":0.0072623640000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"183'2","reported_seq":220,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.061272+0000","last_change":"2024-09-21T14:24:19.061392+0000","last_active":"2024-09-21T14:24:19.061272+0000","last_peered":"2024-09-21T14:24:19.061272+0000","last_clean":"2024-09-21T14:24:19.061272+0000","last_became_active":"2024-09-21T14:23:36.163998+0000","last_became_peered":"2024-09-21T14:23:36.163998+0000","last_unstale":"2024-09-21T14:24:19.061272+0000","last_undegraded":"2024-09-21T14:24:19.061272+0000","last_fullsized":"2024-09-21T14:24:19.061272+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'2","last_scrub_stamp":"2024-09-21T14:24:19.061250+0000","last_deep_scrub":"183'2","last_deep_scrub_stamp":"2024-09-21T14:24:19.061250+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.061250+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:42.514797+0000","scrub_duration":0.0036601849999999998,"objects_trimmed":0,"snaptrim_duration":0.000280597,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"183'816","reported_seq":3558,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.514131+0000","last_change":"2024-09-21T14:24:25.514243+0000","last_active":"2024-09-21T14:24:25.514131+0000","last_peered":"2024-09-21T14:24:25.514131+0000","last_clean":"2024-09-21T14:24:25.514131+0000","last_became_active":"2024-09-21T14:21:18.150122+0000","last_became_peered":"2024-09-21T14:21:18.150122+0000","last_unstale":"2024-09-21T14:24:25.514131+0000","last_undegraded":"2024-09-21T14:24:25.514131+0000","last_fullsized":"2024-09-21T14:24:25.514131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'816","last_scrub_stamp":"2024-09-21T14:24:25.514108+0000","last_deep_scrub":"183'816","last_deep_scrub_stamp":"2024-09-21T14:24:25.514108+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.514108+0000","objects_scrubbed":19,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:36:29.643670+0000","scrub_duration":0.017727863,"objects_trimmed":0,"snaptrim_duration":0.00097190099999999995,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5379,"num_read_kb":124715,"num_write":999,"num_write_kb":60255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"183'3","reported_seq":173,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.928154+0000","last_change":"2024-09-21T14:24:24.928283+0000","last_active":"2024-09-21T14:24:24.928154+0000","last_peered":"2024-09-21T14:24:24.928154+0000","last_clean":"2024-09-21T14:24:24.928154+0000","last_became_active":"2024-09-21T14:23:56.176457+0000","last_became_peered":"2024-09-21T14:23:56.176457+0000","last_unstale":"2024-09-21T14:24:24.928154+0000","last_undegraded":"2024-09-21T14:24:24.928154+0000","last_fullsized":"2024-09-21T14:24:24.928154+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'3","last_scrub_stamp":"2024-09-21T14:24:24.928131+0000","last_deep_scrub":"183'3","last_deep_scrub_stamp":"2024-09-21T14:24:24.928131+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.928131+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:49:14.281753+0000","scrub_duration":0.0038982660000000001,"objects_trimmed":0,"snaptrim_duration":0.00071109000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"183'1264","reported_seq":3029,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.908797+0000","last_change":"2024-09-21T14:24:28.908797+0000","last_active":"2024-09-21T14:24:28.908797+0000","last_peered":"2024-09-21T14:24:28.908797+0000","last_clean":"2024-09-21T14:24:28.908797+0000","last_became_active":"2024-09-21T14:22:53.149875+0000","last_became_peered":"2024-09-21T14:22:53.149875+0000","last_unstale":"2024-09-21T14:24:28.908797+0000","last_undegraded":"2024-09-21T14:24:28.908797+0000","last_fullsized":"2024-09-21T14:24:28.908797+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'1264","last_scrub_stamp":"2024-09-21T14:24:28.908652+0000","last_deep_scrub":"183'1264","last_deep_scrub_stamp":"2024-09-21T14:24:28.908652+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.908652+0000","objects_scrubbed":8,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:33:16.414971+0000","scrub_duration":0.0086359590000000003,"objects_trimmed":0,"snaptrim_duration":0.001301881,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4856,"num_read_kb":16556,"num_write":1705,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6255410,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":140,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:33.555277+0000","last_change":"2024-09-21T14:24:33.555392+0000","last_active":"2024-09-21T14:24:33.555277+0000","last_peered":"2024-09-21T14:24:33.555277+0000","last_clean":"2024-09-21T14:24:33.555277+0000","last_became_active":"2024-09-21T14:22:08.452778+0000","last_became_peered":"2024-09-21T14:22:08.452778+0000","last_unstale":"2024-09-21T14:24:33.555277+0000","last_undegraded":"2024-09-21T14:24:33.555277+0000","last_fullsized":"2024-09-21T14:24:33.555277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:33.555255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:33.555255+0000","last_clean_scrub_stamp":"2024-09-21T14:24:33.555255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:36.660361+0000","scrub_duration":0.001445265,"objects_trimmed":0,"snaptrim_duration":0.00032564900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"183'623","reported_seq":3080,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:23.121148+0000","last_change":"2024-09-21T14:24:23.121272+0000","last_active":"2024-09-21T14:24:23.121148+0000","last_peered":"2024-09-21T14:24:23.121148+0000","last_clean":"2024-09-21T14:24:23.121148+0000","last_became_active":"2024-09-21T14:23:53.164331+0000","last_became_peered":"2024-09-21T14:23:53.164331+0000","last_unstale":"2024-09-21T14:24:23.121148+0000","last_undegraded":"2024-09-21T14:24:23.121148+0000","last_fullsized":"2024-09-21T14:24:23.121148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'623","last_scrub_stamp":"2024-09-21T14:24:23.121124+0000","last_deep_scrub":"183'623","last_deep_scrub_stamp":"2024-09-21T14:24:23.121124+0000","last_clean_scrub_stamp":"2024-09-21T14:24:23.121124+0000","objects_scrubbed":14,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:02.221894+0000","scrub_duration":0.011825186,"objects_trimmed":0,"snaptrim_duration":0.00075621399999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3644,"num_read_kb":12438,"num_write":706,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":58949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:28.571302+0000","last_change":"2024-09-21T14:24:28.571442+0000","last_active":"2024-09-21T14:24:28.571302+0000","last_peered":"2024-09-21T14:24:28.571302+0000","last_clean":"2024-09-21T14:24:28.571302+0000","last_became_active":"2024-09-21T14:23:36.158879+0000","last_became_peered":"2024-09-21T14:23:36.158879+0000","last_unstale":"2024-09-21T14:24:28.571302+0000","last_undegraded":"2024-09-21T14:24:28.571302+0000","last_fullsized":"2024-09-21T14:24:28.571302+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:24:28.571280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:24:28.571280+0000","last_clean_scrub_stamp":"2024-09-21T14:24:28.571280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:14.893355+0000","scrub_duration":0.0019501270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"183'692","reported_seq":3851,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.165956+0000","last_change":"2024-09-21T14:24:29.166130+0000","last_active":"2024-09-21T14:24:29.165956+0000","last_peered":"2024-09-21T14:24:29.165956+0000","last_clean":"2024-09-21T14:24:29.165956+0000","last_became_active":"2024-09-21T14:21:18.146151+0000","last_became_peered":"2024-09-21T14:21:18.146151+0000","last_unstale":"2024-09-21T14:24:29.165956+0000","last_undegraded":"2024-09-21T14:24:29.165956+0000","last_fullsized":"2024-09-21T14:24:29.165956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'692","last_scrub_stamp":"2024-09-21T14:24:29.165932+0000","last_deep_scrub":"183'692","last_deep_scrub_stamp":"2024-09-21T14:24:29.165932+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.165932+0000","objects_scrubbed":7,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:34.075288+0000","scrub_duration":0.0079997139999999998,"objects_trimmed":0,"snaptrim_duration":0.00075467600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5955,"num_read_kb":16099,"num_write":725,"num_write_kb":12339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"183'714","reported_seq":4833,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:27.649445+0000","last_change":"2024-09-21T14:24:27.649552+0000","last_active":"2024-09-21T14:24:27.649445+0000","last_peered":"2024-09-21T14:24:27.649445+0000","last_clean":"2024-09-21T14:24:27.649445+0000","last_became_active":"2024-09-21T14:23:34.160754+0000","last_became_peered":"2024-09-21T14:23:34.160754+0000","last_unstale":"2024-09-21T14:24:27.649445+0000","last_undegraded":"2024-09-21T14:24:27.649445+0000","last_fullsized":"2024-09-21T14:24:27.649445+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'714","last_scrub_stamp":"2024-09-21T14:24:27.649422+0000","last_deep_scrub":"183'714","last_deep_scrub_stamp":"2024-09-21T14:24:27.649422+0000","last_clean_scrub_stamp":"2024-09-21T14:24:27.649422+0000","objects_scrubbed":11,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:45:37.569259+0000","scrub_duration":0.0090103430000000005,"objects_trimmed":0,"snaptrim_duration":0.00141461,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7676,"num_read_kb":84641,"num_write":719,"num_write_kb":55692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":51553050,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"183'493","reported_seq":2386,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:33.791893+0000","last_change":"2024-09-21T14:24:33.792022+0000","last_active":"2024-09-21T14:24:33.791893+0000","last_peered":"2024-09-21T14:24:33.791893+0000","last_clean":"2024-09-21T14:24:33.791893+0000","last_became_active":"2024-09-21T14:23:39.368971+0000","last_became_peered":"2024-09-21T14:23:39.368971+0000","last_unstale":"2024-09-21T14:24:33.791893+0000","last_undegraded":"2024-09-21T14:24:33.791893+0000","last_fullsized":"2024-09-21T14:24:33.791893+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'493","last_scrub_stamp":"2024-09-21T14:24:33.791867+0000","last_deep_scrub":"183'493","last_deep_scrub_stamp":"2024-09-21T14:24:33.791867+0000","last_clean_scrub_stamp":"2024-09-21T14:24:33.791867+0000","objects_scrubbed":12,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:23:56.649520+0000","scrub_duration":0.0099566480000000002,"objects_trimmed":0,"snaptrim_duration":0.001323332,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":1343,"num_read_kb":8616,"num_write":447,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":12049263,"num_keys_recovered":79,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"183'491","reported_seq":2584,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:19.713952+0000","last_change":"2024-09-21T14:24:19.714091+0000","last_active":"2024-09-21T14:24:19.713952+0000","last_peered":"2024-09-21T14:24:19.713952+0000","last_clean":"2024-09-21T14:24:19.713952+0000","last_became_active":"2024-09-21T14:24:11.286380+0000","last_became_peered":"2024-09-21T14:24:11.286380+0000","last_unstale":"2024-09-21T14:24:19.713952+0000","last_undegraded":"2024-09-21T14:24:19.713952+0000","last_fullsized":"2024-09-21T14:24:19.713952+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'491","last_scrub_stamp":"2024-09-21T14:24:19.713929+0000","last_deep_scrub":"183'491","last_deep_scrub_stamp":"2024-09-21T14:24:19.713929+0000","last_clean_scrub_stamp":"2024-09-21T14:24:19.713929+0000","objects_scrubbed":10,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:08.648885+0000","scrub_duration":0.0091389910000000008,"objects_trimmed":0,"snaptrim_duration":0.00044423299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2728,"num_read_kb":10420,"num_write":568,"num_write_kb":7173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"183'755","reported_seq":2644,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:21.875096+0000","last_change":"2024-09-21T14:24:21.875206+0000","last_active":"2024-09-21T14:24:21.875096+0000","last_peered":"2024-09-21T14:24:21.875096+0000","last_clean":"2024-09-21T14:24:21.875096+0000","last_became_active":"2024-09-21T14:21:18.148591+0000","last_became_peered":"2024-09-21T14:21:18.148591+0000","last_unstale":"2024-09-21T14:24:21.875096+0000","last_undegraded":"2024-09-21T14:24:21.875096+0000","last_fullsized":"2024-09-21T14:24:21.875096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'755","last_scrub_stamp":"2024-09-21T14:24:21.875072+0000","last_deep_scrub":"183'755","last_deep_scrub_stamp":"2024-09-21T14:24:21.875072+0000","last_clean_scrub_stamp":"2024-09-21T14:24:21.875072+0000","objects_scrubbed":12,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:49.964907+0000","scrub_duration":0.014754738999999999,"objects_trimmed":0,"snaptrim_duration":0.00053120800000000003,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3998,"num_read_kb":18124,"num_write":915,"num_write_kb":11015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"183'578","reported_seq":3116,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:29.953850+0000","last_change":"2024-09-21T14:24:29.953981+0000","last_active":"2024-09-21T14:24:29.953850+0000","last_peered":"2024-09-21T14:24:29.953850+0000","last_clean":"2024-09-21T14:24:29.953850+0000","last_became_active":"2024-09-21T14:23:53.169428+0000","last_became_peered":"2024-09-21T14:23:53.169428+0000","last_unstale":"2024-09-21T14:24:29.953850+0000","last_undegraded":"2024-09-21T14:24:29.953850+0000","last_fullsized":"2024-09-21T14:24:29.953850+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'578","last_scrub_stamp":"2024-09-21T14:24:29.953817+0000","last_deep_scrub":"183'578","last_deep_scrub_stamp":"2024-09-21T14:24:29.953817+0000","last_clean_scrub_stamp":"2024-09-21T14:24:29.953817+0000","objects_scrubbed":8,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:04:53.938190+0000","scrub_duration":0.0068265540000000003,"objects_trimmed":0,"snaptrim_duration":0.00094992299999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3640,"num_read_kb":14541,"num_write":708,"num_write_kb":16274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":5893860,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"183'1009","reported_seq":3936,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:24.137471+0000","last_change":"2024-09-21T14:24:24.137641+0000","last_active":"2024-09-21T14:24:24.137471+0000","last_peered":"2024-09-21T14:24:24.137471+0000","last_clean":"2024-09-21T14:24:24.137471+0000","last_became_active":"2024-09-21T14:23:53.168290+0000","last_became_peered":"2024-09-21T14:23:53.168290+0000","last_unstale":"2024-09-21T14:24:24.137471+0000","last_undegraded":"2024-09-21T14:24:24.137471+0000","last_fullsized":"2024-09-21T14:24:24.137471+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'1009","last_scrub_stamp":"2024-09-21T14:24:24.137430+0000","last_deep_scrub":"183'1009","last_deep_scrub_stamp":"2024-09-21T14:24:24.137430+0000","last_clean_scrub_stamp":"2024-09-21T14:24:24.137430+0000","objects_scrubbed":11,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:01:18.332693+0000","scrub_duration":0.011649343,"objects_trimmed":2,"snaptrim_duration":0.0034383339999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6100,"num_read_kb":20639,"num_write":1427,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":10946810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"183'530","reported_seq":5185,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:27.981932+0000","last_change":"2024-09-21T14:24:27.982058+0000","last_active":"2024-09-21T14:24:27.981932+0000","last_peered":"2024-09-21T14:24:27.981932+0000","last_clean":"2024-09-21T14:24:27.981932+0000","last_became_active":"2024-09-21T14:23:56.176993+0000","last_became_peered":"2024-09-21T14:23:56.176993+0000","last_unstale":"2024-09-21T14:24:27.981932+0000","last_undegraded":"2024-09-21T14:24:27.981932+0000","last_fullsized":"2024-09-21T14:24:27.981932+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'530","last_scrub_stamp":"2024-09-21T14:24:27.981907+0000","last_deep_scrub":"183'530","last_deep_scrub_stamp":"2024-09-21T14:24:27.981907+0000","last_clean_scrub_stamp":"2024-09-21T14:24:27.981907+0000","objects_scrubbed":13,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:05.795907+0000","scrub_duration":0.015791745999999999,"objects_trimmed":0,"snaptrim_duration":0.001354482,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5956,"num_read_kb":25984,"num_write":472,"num_write_kb":21468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6425791,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"183'454","reported_seq":3615,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:39.522319+0000","last_change":"2024-09-21T14:24:39.522399+0000","last_active":"2024-09-21T14:24:39.522319+0000","last_peered":"2024-09-21T14:24:39.522319+0000","last_clean":"2024-09-21T14:24:39.522319+0000","last_became_active":"2024-09-21T14:23:39.370670+0000","last_became_peered":"2024-09-21T14:23:39.370670+0000","last_unstale":"2024-09-21T14:24:39.522319+0000","last_undegraded":"2024-09-21T14:24:39.522319+0000","last_fullsized":"2024-09-21T14:24:39.522319+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'454","last_scrub_stamp":"2024-09-21T14:24:39.522297+0000","last_deep_scrub":"183'454","last_deep_scrub_stamp":"2024-09-21T14:24:39.522297+0000","last_clean_scrub_stamp":"2024-09-21T14:24:39.522297+0000","objects_scrubbed":14,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:06.713056+0000","scrub_duration":0.0088792019999999992,"objects_trimmed":0,"snaptrim_duration":0.00065706300000000005,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3011,"num_read_kb":16367,"num_write":383,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":5074577,"num_keys_recovered":25,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'401","reported_seq":3392,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:26.668381+0000","last_change":"2024-09-21T14:24:26.668505+0000","last_active":"2024-09-21T14:24:26.668381+0000","last_peered":"2024-09-21T14:24:26.668381+0000","last_clean":"2024-09-21T14:24:26.668381+0000","last_became_active":"2024-09-21T14:23:52.162961+0000","last_became_peered":"2024-09-21T14:23:52.162961+0000","last_unstale":"2024-09-21T14:24:26.668381+0000","last_undegraded":"2024-09-21T14:24:26.668381+0000","last_fullsized":"2024-09-21T14:24:26.668381+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'401","last_scrub_stamp":"2024-09-21T14:24:26.668357+0000","last_deep_scrub":"183'401","last_deep_scrub_stamp":"2024-09-21T14:24:26.668357+0000","last_clean_scrub_stamp":"2024-09-21T14:24:26.668357+0000","objects_scrubbed":18,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:09:57.562717+0000","scrub_duration":0.012675017,"objects_trimmed":0,"snaptrim_duration":0.0013772509999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2074,"num_read_kb":23209,"num_write":413,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":27926980,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"183'561","reported_seq":3801,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:25.928031+0000","last_change":"2024-09-21T14:24:25.928166+0000","last_active":"2024-09-21T14:24:25.928031+0000","last_peered":"2024-09-21T14:24:25.928031+0000","last_clean":"2024-09-21T14:24:25.928031+0000","last_became_active":"2024-09-21T14:23:33.164730+0000","last_became_peered":"2024-09-21T14:23:33.164730+0000","last_unstale":"2024-09-21T14:24:25.928031+0000","last_undegraded":"2024-09-21T14:24:25.928031+0000","last_fullsized":"2024-09-21T14:24:25.928031+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'561","last_scrub_stamp":"2024-09-21T14:24:25.928006+0000","last_deep_scrub":"183'561","last_deep_scrub_stamp":"2024-09-21T14:24:25.928006+0000","last_clean_scrub_stamp":"2024-09-21T14:24:25.928006+0000","objects_scrubbed":12,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:18.286938+0000","scrub_duration":0.0097054910000000001,"objects_trimmed":0,"snaptrim_duration":0.00084428400000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4896,"num_read_kb":22999,"num_write":573,"num_write_kb":10781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":5533980,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"183'1134","reported_seq":4910,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-21T14:24:41.778494+0000","last_change":"2024-09-21T14:24:41.778640+0000","last_active":"2024-09-21T14:24:41.778494+0000","last_peered":"2024-09-21T14:24:41.778494+0000","last_clean":"2024-09-21T14:24:41.778494+0000","last_became_active":"2024-09-21T14:23:35.170597+0000","last_became_peered":"2024-09-21T14:23:35.170597+0000","last_unstale":"2024-09-21T14:24:41.778494+0000","last_undegraded":"2024-09-21T14:24:41.778494+0000","last_fullsized":"2024-09-21T14:24:41.778494+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'1134","last_scrub_stamp":"2024-09-21T14:24:41.778472+0000","last_deep_scrub":"183'1134","last_deep_scrub_stamp":"2024-09-21T14:24:41.778472+0000","last_clean_scrub_stamp":"2024-09-21T14:24:41.778472+0000","objects_scrubbed":9,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:04:19.479173+0000","scrub_duration":0.0088559339999999993,"objects_trimmed":0,"snaptrim_duration":0.001449842,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7860,"num_read_kb":25015,"num_write":1484,"num_write_kb":14107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":13563992,"num_keys_recovered":93,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":380,"num_read":149021,"num_read_kb":954839,"num_write":21859,"num_write_kb":503763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6231,"num_bytes_recovered":282785721,"num_keys_recovered":699,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19830,"ondisk_log_size":19830,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100500,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55412,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316428,"statfs":{"total":96636764160,"available":96580022272,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.434},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.61899999999999999,"5min":0.749,"15min":0.749},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.67700000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66400000000000003}]},{"osd":1,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46700000000000003,"15min":0.45800000000000002},"min":{"1min":0.33000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.435},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.624}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.441,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.627,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.747}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.68100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81200000000000006,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.373,"15min":0.37},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.61699999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42699999999999999},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.42299999999999999},"min":{"1min":0.32700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66700000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.35099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.621,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.41599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46000000000000002},"min":{"1min":0.36499999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.32500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.27800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.57599999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0189999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59899999999999998,"15min":0.65500000000000003},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.68899999999999995}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.253,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63500000000000001,"5min":0.747,"15min":0.747},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44600000000000001,"15min":0.45100000000000001},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66800000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51300000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60799999999999998,"5min":0.747,"15min":0.747},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.47699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46999999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59572,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312268,"statfs":{"total":96636764160,"available":96575762432,"internally_reserved":0,"allocated":2273280,"data_stored":1667371,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.627}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.316,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72599999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68700000000000006,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.61099999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42599999999999999,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67900000000000005}]},{"osd":3,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47799999999999998,"15min":0.47099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63800000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":2.593,"15min":2.593},"last":0.53800000000000003}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.86199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.66000000000000003,"15min":0.753},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":6.7430000000000003,"15min":6.7430000000000003},"last":0.498}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.48799999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59512,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.436},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.73799999999999999,"5min":0.746,"15min":0.746},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.27600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.61099999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":0.71499999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55000000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.374},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.437,"15min":0.443},"min":{"1min":0.26400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.39100000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.41399999999999998}]},{"osd":7,"last update":"Sat Sep 21 14:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.499},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.0109999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.45200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2256896,"data_stored":1668338,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.312,"15min":0.311},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.307},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.34699999999999998},"min":{"1min":0.25700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46400000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.40699999999999997}]},{"osd":1,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.21099999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.372,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.433},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.23000000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45600000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.498,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69599999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79500000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.62},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67400000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.72399999999999998}]},{"osd":6,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.752,"5min":1.714,"15min":1.714},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76800000000000002,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":1.319,"15min":1.319},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29172,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342668,"statfs":{"total":96636764160,"available":96606892032,"internally_reserved":0,"allocated":2207744,"data_stored":1077970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.20799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.433,"5min":0.42899999999999999,"15min":0.45900000000000002},"min":{"1min":0.24099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.466,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.502}]},{"osd":1,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.376,"15min":0.36699999999999999},"min":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.56399999999999995}]},{"osd":3,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.436,"15min":0.46400000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.6100000000000001,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.429,"5min":2.097,"15min":2.097},"last":0.58699999999999997}]},{"osd":4,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.55000000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.747},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86299999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2207744,"data_stored":1077003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.311,"15min":0.30599999999999999},"min":{"1min":0.20499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.21199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.41099999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.36199999999999999,"15min":0.35799999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38}]},{"osd":3,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.439},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.49199999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.44900000000000001},"min":{"1min":0.33200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51300000000000001}]},{"osd":5,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.496}]},{"osd":7,"last update":"Sat Sep 21 14:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.373,"15min":0.373},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65272,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306568,"statfs":{"total":96636764160,"available":96569925632,"internally_reserved":0,"allocated":2211840,"data_stored":1077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.34000000000000002,"15min":0.35399999999999998},"min":{"1min":0.17899999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.45400000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36299999999999999,"15min":0.37},"min":{"1min":0.28100000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.55800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33400000000000002,"15min":0.34799999999999998},"min":{"1min":0.245,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53200000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.34100000000000003,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34499999999999997}]},{"osd":3,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38200000000000001,"15min":0.40600000000000003},"min":{"1min":0.214,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53500000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.443,"15min":0.46600000000000003},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55300000000000005,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59399999999999997,"5min":1.292,"15min":1.292},"last":0.502},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.59399999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":1.341,"15min":1.341},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.36099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63}]},{"osd":6,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":1.284,"15min":1.284},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Sat Sep 21 14:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74299999999999999,"5min":1.254,"15min":1.254},"last":0.56799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32410,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:24:59.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T14:24:59.920 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T14:24:59.920 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T14:24:59.920 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T14:24:59.920 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:24:59.958 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T14:24:59.958 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T14:24:59.958 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.028 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T14:25:00.028 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T14:25:00.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.099 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T14:25:00.099 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T14:25:00.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.179 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T14:25:00.179 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T14:25:00.179 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.217 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T14:25:00.217 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T14:25:00.217 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.290 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T14:25:00.290 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T14:25:00.290 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.364 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T14:25:00.364 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T14:25:00.364 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.436 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T14:25:00.436 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T14:25:00.436 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T14:25:00.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.462 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T14:25:00.462 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T14:25:00.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-21T14:25:00.503 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T14:25:00.503 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T14:25:00.503 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T14:25:00.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.514 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T14:25:00.514 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T14:25:00.514 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.566 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T14:25:00.566 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T14:25:00.566 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T14:25:00.578 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T14:25:00.578 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T14:25:00.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-21T14:25:00.636 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:25:00.636 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T14:25:00.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:25:00.881 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T14:25:00.960 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:25:00.961 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T14:25:01.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:25:01.017 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T14:25:01.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:25:01.069 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T14:25:01.197 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:25:01.197 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T14:25:01.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:25:01.279 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T14:25:01.340 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:25:01.340 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T14:25:01.398 INFO:tasks.ceph:Archiving mon data... 2024-09-21T14:25:01.399 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/data/mon.a.tgz 2024-09-21T14:25:01.399 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-21T14:25:01.415 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.UWHRZHRnZs 2024-09-21T14:25:01.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.UWHRZHRnZs 2024-09-21T14:25:01.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.UWHRZHRnZs 2024-09-21T14:25:01.657 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.UWHRZHRnZs is 406KB 2024-09-21T14:25:01.752 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.UWHRZHRnZs 2024-09-21T14:25:01.767 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/data/mon.c.tgz 2024-09-21T14:25:01.767 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-21T14:25:01.821 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.c8UvKH6sQV 2024-09-21T14:25:01.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.c8UvKH6sQV 2024-09-21T14:25:02.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.c8UvKH6sQV 2024-09-21T14:25:02.082 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.c8UvKH6sQV is 553KB 2024-09-21T14:25:02.180 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.c8UvKH6sQV 2024-09-21T14:25:02.195 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/data/mon.b.tgz 2024-09-21T14:25:02.195 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-09-21T14:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.9mo35cm7Nt 2024-09-21T14:25:02.214 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9mo35cm7Nt 2024-09-21T14:25:02.404 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.9mo35cm7Nt 2024-09-21T14:25:02.487 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.9mo35cm7Nt is 553KB 2024-09-21T14:25:02.558 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.9mo35cm7Nt 2024-09-21T14:25:02.573 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T14:25:02.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T14:25:02.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T14:25:02.636 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-21T14:25:02.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T14:25:02.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T14:25:02.664 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T14:25:02.704 INFO:teuthology.orchestra.run.smithi161.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T14:25:02.708 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T14:25:02.710 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi073/crash 2024-09-21T14:25:02.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T14:25:02.735 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi161/crash 2024-09-21T14:25:02.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T14:25:02.773 INFO:tasks.ceph:Compressing logs... 2024-09-21T14:25:02.773 DEBUG:teuthology.orchestra.run.smithi073:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T14:25:02.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T14:25:05.504 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-21T14:25:12.493 INFO:tasks.ceph:Archiving logs... 2024-09-21T14:25:12.493 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi073/log 2024-09-21T14:25:12.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T14:25:13.708 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi161/log 2024-09-21T14:25:13.709 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T14:25:14.682 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T14:25:14.690 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T14:25:14.691 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T14:25:14.693 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T14:25:14.783 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T14:25:14.792 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T14:25:14.792 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:25:14.795 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:25:14.810 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-21T14:25:14.811 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-21T14:25:14.814 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T14:25:14.814 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-21T14:25:14.814 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +867us[ +865us] +/- 54ms 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -3132us[-3134us] +/- 76ms 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +1627us[+1627us] +/- 61ms 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-21T14:25:14.815 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +867us[ +867us] +/- 54ms 2024-09-21T14:25:14.816 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -3141us[-3141us] +/- 75ms 2024-09-21T14:25:14.816 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +1638us[+1638us] +/- 61ms 2024-09-21T14:25:14.816 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T14:25:14.817 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T14:25:14.825 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T14:25:14.825 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T14:25:14.833 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T14:25:14.941 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T14:25:15.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T14:25:15.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:25:15.050 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T14:25:15.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:25:15.079 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T14:25:15.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T14:25:15.096 INFO:teuthology.task.internal:Duration was 649.990972 seconds 2024-09-21T14:25:15.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T14:25:15.103 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T14:25:15.103 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T14:25:15.106 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T14:25:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T14:25:15.159 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T14:25:15.357 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T14:25:15.357 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-09-21T14:25:15.357 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T14:25:15.417 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T14:25:15.417 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T14:25:15.444 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T14:25:15.444 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T14:25:15.459 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T14:25:15.511 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T14:25:15.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T14:25:15.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T14:25:15.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T14:25:15.683 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T14:25:15.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T14:25:15.707 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T14:25:15.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T14:25:15.742 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T14:25:15.750 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T14:25:15.771 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-09-21T14:25:15.795 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-09-21T14:25:15.836 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T14:25:15.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:25:15.862 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T14:25:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:25:15.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T14:25:15.901 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T14:25:15.901 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi073 2024-09-21T14:25:15.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T14:25:15.941 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909929/remote/smithi161 2024-09-21T14:25:15.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T14:25:15.975 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T14:25:15.976 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T14:25:15.978 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T14:25:16.025 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T14:25:16.036 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T14:25:16.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T14:25:16.044 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T14:25:16.045 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T14:25:16.047 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T14:25:16.059 INFO:teuthology.orchestra.run.smithi073.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 14:25 /home/ubuntu/cephtest 2024-09-21T14:25:16.082 INFO:teuthology.orchestra.run.smithi161.stdout: 265933 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 14:25 /home/ubuntu/cephtest 2024-09-21T14:25:16.082 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T14:25:16.092 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T14:25:16.111 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} duration: 649.9909722805023 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T14:25:16.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T14:25:16.219 INFO:teuthology.run:pass